Setup ODK-X Sync Endpoint Manually

Follow these setup instructions for manual or local setup of the sync endpoint. If deploying to a cloud service, check out Cloud-based Setup

Prerequisites

You must have Docker 18.09.2 or newer, and be running in Swarm Mode. Follow these links for detailed instructions on installing Docker and enabling Swarm Mode.

If you wish to enable HTTPS, you also need to install certbot

ODK-X Sync Endpoint Setup

ODK-X Sync Endpoint requires a database and a LDAP directory, you could follow the instructions and deploy all three components together or supply your own database and/or LDAP directory.

Note

All of the following commands should be run on your server.

If you are using git on Windows, make sure git is configured with "core.autocrlf=false" - otherwise it will convert line endings with LF to CRLF, which will cause problems with the .sh-files when used in the Docker containers, thus preventing odk/sync-endpoint from starting and instead just returning with an ":invalid argument"-error.

Setup instructions:

  1. Choose a directory to store you endpoint in. In that directory, run:

$ git clone https://github.com/odk-x/sync-endpoint-default-setup
  1. Navigate into the the "sync-endpoint-default-setup" directory

  2. Checkout the sync-endpoint code by running:

$ git clone https://github.com/odk-x/sync-endpoint
  1. Navigate into the sync-endpoint directory. Most likely

$ cd sync-endpoint
  1. Build sync endpoint by running the following: (NOTE: you will need Apache Maven installed >= 3.3.3)

$ mvn clean install
  1. Navigate back to the parent "sync-endpoint-default-setup" directory.

  2. In the "sync-endpoint-default-setup" directory run:

$ docker build --pull -t odk/sync-web-ui https://github.com/odk-x/sync-endpoint-web-ui.git
  1. In the "sync-endpoint-default-setup" cloned repository run:

$ docker build --pull -t odk/db-bootstrap db-bootstrap
  1. In the "sync-endpoint-default-setup" cloned repository run:

$ docker build --pull -t odk/openldap openldap
  1. In the "sync-endpoint-default-setup" cloned repository run:

$ docker build --pull -t odk/phpldapadmin phpldapadmin
  1. Enter your hostname in the security.server.hostname field in the security.properties file (under the directory config/sync-endpoint).

  2. If you're not using the standard ports (80 for HTTP and 443 for HTTPS) enter the ports you're using in the security.server.port and security.server.securePort fields in the security.properties. Then edit the ports section under the sync section in docker-compose.yml to be YOUR_PORT:8080.

Note

It is important that the right side of the colon stays as 8080. This is the internal port that the web server is looking for.

  1. If you're using your own LDAP directory or database, continue with the instructions:

  1. In the "sync-endpoint-default-setup" cloned repository run:

  • For HTTP:

$ docker stack deploy -c docker-compose.yml syncldap
  • For HTTPS:

$ docker stack deploy -c docker-compose.yml -c docker-compose-https.yml syncldap
  1. The server takes about 30s to start, then it will be running at http://127.0.0.1.

  2. See the LDAP section for instructions on configuring users and groups.

Custom database

  1. If you haven't followed the common instructions, start with those.

  2. Remove the db and db-bootstrap sections in docker-compose.yml.

  3. Modify jdbc.properties to match your database. Supported database systems are PostgreSQL, MySQL and Microsoft SQL Server. Sample config for each type of database can be found on Github.

  4. Modify sync.env to match your database

  5. In the cloned repository,

$ docker stack deploy -c docker-compose.yml syncldap
  1. The server takes about 30s to start, then it will be running at http://127.0.0.1.

Custom LDAP directory

  1. If you haven't followed the common instructions, start with those.

  2. OPTIONAL: If your LDAP directory uses a certificate that was signed by a self-signed CA,

  1. Make the public key of the CA available to ODK-X Sync Endpoint with this command.

$ docker config create org.opendatakit.sync.ldapcert PATH_TO_CERT
  1. Uncomment the relevant lines in the configs section in docker-compose.yml and the configs section under the sync section in docker-compose.yml.

  1. Remove the ldap-service and phpldapadmin sections in docker-compose.yml.

  2. Modify the relevant sections in security.properties to match your LDAP directory. Further instructions are in the file.

Note

The default configuration does not use ldaps or StartTLS because the LDAP directory communicates with the ODK-X Sync Endpoint over a secure overlay network. You should use ldaps or StartTLS to communicate with your LDAP directory.

  1. In the cloned repository:

$ docker stack deploy -c docker-compose.yml syncldap
  1. The server takes about 30s to start, then it will be running at http://127.0.0.1.

Stopping ODK-X Sync Endpoint

  1. Run:

$ docker stack rm syncldap
  1. OPTIONAL: If you want to remove the volumes as well,

Warning

Removing volumes will remove any provisioned TLS keys if https is enabled. These keys can only be provisioned at a rate of 50 valid keys/domain/week.

  • Linux/macOS:

$ docker volume rm $(docker volume ls -f "label=com.docker.stack.namespace=syncldap" -q)
  • Windows:

$ docker volume rm (docker volume ls -f "label=com.docker.stack.namespace=syncldap" -q)