Bootstrapping a VM for development

This document describes the steps for creating your own Ubuntu based VM on which all components are installed and ready to host a local version for your development purposes.

NOTE: For registered partners of the project we offer a free managed VM.

Choosing a VM provider

You can either setup a local VM using your favourite method or choose an online provider. Make sure you get at least two cores and at least 4G of RAM. Any amount of storage more than 10G should be fine.

Choose the distro

Instructions below are assuming you are using the recent version of Ubuntu 18.04. Other versions of Linux might be fine too but you need to bring your own support :) Linux is one of the supported platforms of Neo4J, macOS and Windows being the other two. Any of these would, ion theory, enable you to run the complete stack, however the backend is never tested on Windows.

Install the bistro as needed, optionally set the hostname with:

sudo hostnamectl set-hostname your_hostname

Assuming you now have a fresh install of Ubuntu 18.04 we follow this recipe:

Install Neo4j

Add the repository of Neo4J to the system:

wget -O - https://debian.neo4j.org/neotechnology.gpg.key | sudo apt-key add -
echo 'deb https://debian.neo4j.org/repo stable/' | sudo tee /etc/apt/sources.list.d/neo4j.list

Get the repo info:

apt-get update

Install the current version of the database:

apt-get install neo4j-enterprise

or

apt-get install neo4j

For the community version of Neo4J; certainly for development purposes this should be ok. Currently we are using the current version in the 3.5 branch.

To get rid of some annoying startup error (the dir is not there initially):

mkdir /var/run/neo4j
chown -R neo4j /var/run/neo4j

Update the limit settings with:

vi /etc/default/neo4j

and remove the comment in front of NEO4J_ULIMIT_NOFILE=40000

Install APOC (make sure you use version matching the neo4j installed). Some parts of the API make use of APOC for their functionality.

cd /var/lib/neo4j/plugins/
wget https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.5.0.4/apoc-3.5.0.4-all.jar

Once everything is installed we can enable and start the database server:

systemctl enable neo4j
systemctl restart neo4j

Next we need to install Python and tools to run the backend server: As we are using the async features Python 3.7 is recommended but 3.6 should work fine too. Other versions are not supported!

apt install build-essential
apt install python3.7-dev

(apt instal python3.7) (TODO: check if we need to!)

apt install python3-pip 

This installs pip for the system default python (which is 3.6 on Ubuntu 18.04) but we can use it with our own python3.7 as well by specifying it as such:


python3.7 -m pip install setuptools
python3.7 -m pip install wheel
python3.7 -m pip install sanic
python3.7 -m pip install py2neo
python3.7 -m pip install rollbar
python3.7 -m pip install python-pptx
python3.7 -m pip install python-slugify
python3.7 -m pip install dnspython

This avoids the mess which is setting up pip for a non-default Python interpreter.

In future we will change the setup to a [venv](https://docs.python.org/3/library/venv.html)

Get the repositories and install them in the location they except to be in:

mkdir -p /web/sites/historiana.eu/api
mkdir -p /web/sites/historiana.eu/site
cd /web/sites/historiana.eu/api
git clone https://gitlab.com/historiana/api.git

To enable the automatic start of the API on reboot we need to create a service which is run at startup.

Create a textile as /lib/systemd/system/hiapi.service with these contents:

[Unit]
Description=Historiana API
After=syslog.target
After=network.target


[Service]
Type=simple
User=www-data
Group=daemon
ExecStart=/usr/bin/python3 /web/sites/historiana.eu/api/server.py
WorkingDirectory=/web/sites/historiana.eu/api/


# Give the script some time to startup
TimeoutSec=300


[Install]
WantedBy=multi-user.target

Then enable and install this service (similar to Neo4J earlier)

systemctl enable hiapi
systemctl restart hiapi

If you want to use SSL you will need a certificate for your instance. In order to get this your hostname should resolve to the IP of the VM.

Using certbot you can get a free Lets Encrypt certificate:

apt install certbot
certbot certonly —standalone -d dev.historiana.eu 
certbot certonly —standalone -d dev.hi.st
openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

Before we can access the site we need to install the webserver NGiNX :

apt install nginx

Then install the supplied dev.historiana.conf in /etc/nginx/sites-enabled

This concludes the installation of the backend, installing the frontend as well is a logical next step and will be documented later.

** TODO: document Frontend install