Clojure & Datalog is Lisp & Prolog

2022-01-13

PERMALINK

IntelliJ IDEA and Cursive

Datalog Exercise in IntelliJ IDEA / Cursive IDE.

Introduction

I started my software engineering studies in 1991 at the Helsinki University of Technology (nowadays Aalto University). I did my Master’s thesis at the Artificial Intelligence department, so I had a chance to do some Lisp and Prolog programming during my studies. When I graduated, my first job was a C programmer job. At that time, it was impossible to find a programmer job for Lisp or Prolog outside academia. I couldn’t have dreamed me programming Lisp and Prolog professionally one day.

Datomic and Datalog

About one year ago, I got interested in Datomic, a transactional database with a flexible data model that uses Datalog as a query language. I implemented my Simple Server exercise using Datomic as data storage; you can read more about that project in Clojure Datomic Exercise. Then one year passed, and I had forgotten Datomic. In December 2021, I started a new project where I implemented some queries against a Datomic database. I got excited regarding Datomic’s query model. Datalog is a subset of Prolog, i.e., you make the queries as logic programming against facts in a database.

Clojure and Datalog is Lisp and Prolog

Lisp is one of the oldest programming languages. Lisp quickly established itself as the favored programming language for artificial intelligence (AI) research since Lisp suits particularly well for symbol processing. Today there are many Lisp dialects, Clojure being one of them. Clojure runs on top of JVM and can be used as a general-purpose programming language. Clojure is a functional programming language with a strong emphasis on immutable data structures and data-intensive processing - Clojure is actually an excellent tool for traditional enterprise software as well.

Prolog is a logic programming language. In a Prolog program, there are certain facts and rules interpreting those facts. Prolog is well-suited for specific tasks that benefit from rule-based logical queries.

As Wikipedia says, Datalog is a declarative logic programming language that syntactically is a subset of Prolog. It is often used as a query language for deductive databases. The Datomic database uses Datalog as a query language.

So, after some 30 years, I finally write Lisp and Prolog professionally. :-)

Learn Datalog Today

I wanted to learn Datalog more. I found an excellent website with good introductory exercises for learning Datalog: Learn Datalog Today. The website provides an HTML interface for writing answers to the exercises. Since using HTML as an editor is a bit clumsy, I downloaded the Movies database schema and facts from the project’s Github repository.

I had already installed the Datomic local development database. Then I did some minor processing and transacted the Movies schema and the facts to the Datomic local development database:

(ns movies
  (:require [clojure.edn :as edn]
            [datomic.client.api :as d]
            [datomic.api :as dapi]
            [potpuri.core :as p]
            [hashp.core]))

(def cfg {:server-type :dev-local
          :storage-dir :mem
          :system "movies"})

(def client (d/client cfg))

(def movies-schema (-> (slurp "resources/movies-schema.edn")
                       (edn/read-string)))

; Quick hack to convert legacy db/id to string format.
(defn db-id-reader [[k l]]
  (str l))

(def movies-data (let [raw (slurp "resources/movies.edn")
                       data (edn/read-string {:readers {'db/id db-id-reader
                                                        'db/fn datomic.function/construct
                                                        'base64 datomic.codec/base-64-literal}} raw)]
                   data))

(defn init-db []
  (let [_ (d/delete-database client {:db-name "movies"})
        _ (d/create-database client {:db-name "movies"})
        conn (d/connect client {:db-name "movies"})
        schema-ret (d/transact conn {:tx-data movies-schema})
        db (:db-after (d/transact conn {:tx-data movies-data}))]
    db))

(def db (init-db))

This way, I have a nice development bench for the exercises: I can write the datalog queries using my favorite Clojure IDE, Cursive on IntelliJ IDEA.

I saved the transacted database into db var. This way creating the answers to exercises was quite simple:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Chapter 0
;; http://www.learndatalogtoday.org/chapter/0

;; Find all movies titles in the database.
(d/q '[:find ?title
         :where [_ :movie/title ?title]]
       db)
...

There are nine chapters (0-8) in the Learn Datalog Today website:

0. Extensible Data Notation
1. Basic Queries
2. Data Patterns
3. Parameterized Queries
4. More Queries
5. Predicates
6. Transformation Functions
7. Aggregates
8. Rules

Suppose you are interested to learn Datalog and logic programming. In that case, I strongly recommend you install the Datomic local development database, clone the Learn Datalog Today Github repository, and set up an exercise development bench to create the answers for the exercises and experiment with Datalog. Read each chapter with thought, and then do all exercises at the end of the chapter using your development bench.

Conclusions

Clojure and Prolog make an excellent pair for data-intensive processing. I’d love to implement a real production system using these tools.

The writer is working at Metosin using Clojure in cloud projects. If you are interested in starting a cloud or Clojure project in Finland or getting cloud or Clojure training in Finland, you can contact me by sending an email to my Metosin email address or contact me via LinkedIn.

Kari Marttila