Historiana Developer and Project documentation

Backend Documentation

Introduction

The Historiana backend provides services to the Frontend. It is implemented in Python 3 and uses various third party libraries to create an environment where it can offer services via a HTTP GET or POST request. All permanent storage goes through this API and both the server filesystem and the Neo4J Graph database are used to store data.

API

The functionality of the API can be split in service areas:

Currently the code carries quite a bit of age as the API has been carried over onto new versions of the site. Cleanup will be done as part of the documentation process.

Neo4J

The secret weapon for the project is the Graph database, early in the project we found that relational databases are actually quite bad at storing relations. They tend to re-create them at every query. This caused problems for the "real world abstraction" which Historiana tries to be. The real world is less predictable than a mathematical model and it tends to have information which are hard to keep in rows and columns.

After evaluating various "No-SQL" systems we discovered the graph databases. The Neo4J 1.9 was more a Java library than a server hence we evaluated others like OrientDB and ArangoDB. During this period Neo released version 2.0 which brought a web-interface along the Cypher query language.

This openend the way to an environment in which you could easily experiment with the Graph model without the need to code in Java. While reading up on the model theory O'Reilly Graph Databases I found that the Amsterdam meetup was a very warm environment. Seeing how others applied the made it easy for the new way of thinking click in place.

Years later I found that the actual inspiration to build Neo4J was the development of a content management system. I can vouch for that; the model is perfect for managing an evolving and growing set of casually interconnected data.

Python

We use Python3 with the Sanic framework to implement the API server. The Sanic library is quite recent and is one of the first frameworks which use the new async features. This requires Python 3.5 or better. We tend to use the most recent version available on our platform. At the moment this is the 3.7 branch.

Apart from Sanic we use various other modules, one of the historical artefacts is the lack of version management for the Python part of things. We are not using a virtual environment at the moment. Therefore we lack a complete overview of all modules used. We will implement pipenv as soon as possible.

An incomplete list: