Overview of Deployment Process for Remote Servers
Overview of Deployment Process for Remote Servers
Introduction
The process for updating the Touchscreen and Back Office applications is known as the deployment process. The deployment process is only used for updating applications and databases on remote servers. It is completely separate and independent from the "copy down" process that is used for the initial installation of applications on remote servers.
What does the deployment process actually do?
The following list of actions is paraphrased from Remote Deployment Process - Java:
- Obtains a lock on a file to stop deployment from running more than once.
- Figures out which files/scripts need to be downloaded.
- Confirms that the updates in this deployment haven't already been applied.
- Downloads files/scripts needed for this deployment.
- Confirms that file/scripts were downloaded completely and correctly.
- Backs up the database.
- Executes SQL scripts to update the database.
- Installs new WAR files for each application.
- Restarts Tomcat to "activate" new WAR files.
- Confirms that applications are up and have the correct version numbers.
What are the pieces involved in the deployment process?
- Remote Connections Application
- SSH Keys and Authentication Agent
- Ansible Inventory Script
- Ansible Commands
- Ansible Playbooks
- Deployment Tool
- Shell Scripts
Diagram of pieces involved in deployment process
The following diagram shows the relationships between different pieces of the deployment process. There are three different "tiers" in the diagram indicated by rounded gray boxes:
- Top tier: The Systems Administrator's computer.
- Middle tier: The "admin" server (currently "admin.portal.fullcount.net").
- Bottom tier: The remote server where applications are being deployed.
Arrows generally indicate which piece of the process initiated communication with other pieces of the process. In other words, arrows are not used to show which direction data is flowing; in most cases, data flows both ways. The diagram is not exactly correct (a few arrows could be removed, and several more could be added), but it's pretty close.
What does each piece of the deployment process do?
Remote Connections Application
- Also known as “at-remote-connection”.
- Web interface for creating and managing SSH tunnels to remote devices (servers, tablets, and terminals).
- Provides basic web service:
- Remote devices poll to see if a connection has been requested.
- Remote devices submit TCP port numbers used when SSH tunnels have been created.
SSH Keys and Authentication Agent
- SSH key pairs are used for logging into remote servers.
- Authentication agent caches decrypted private key in memory on your computer.
- Ansible uses keys cached by your authentication agent to connect to remote servers via SSH.
- Currently using Pageant authentication agent from PuTTY.
Ansible Inventory Script
- Queries the database to build an in-memory data structure that includes serial numbers, and TCP port numbers.
- It is HIGHLY dependent on the structure of the “fc_obj_admin” schema.
- Data structure groups remote servers by:
- Timezone
- Community Name
- FullCount role (Primary DB Server, Primary App Server, TS Client, etc.)
- Resident Portal
- Format of this data structure is NOT arbitrary, and it’s not very well documented.
- Format is roughly JSON, but it only supports 2 levels of hierarchy.
- You can put a group in a group (2 levels), but a group in a group in group (3 levels) explodes.
- The magic is the “_meta” group; this is what allows ansible to use Remote Connections.
Ansible Commands
- ansible: Run simple, ad-hoc commands on target hosts.
- ansible-playbook: Apply state from a YAML file called a playbook.
Ansible Playbooks
deployment.yml
- Builds a list of targets based on the expression after “hosts:” in "deployment.yml".
- Confirms SSH port number received from ansible inventory script is not blank.
- Checks for an open TCP connection port number received from ansible inventory script.
- Connects to each remote server via SSH.
- Checks that the serial number of the remote server it connected to matches the serial number it received from ansible inventory script.
- Starts the deployment tool (deployment.jar) on each remote server.
Deployment Tool
deployment.jar
- Command line application installed on remote servers that downloads and executes updates.
- Documented in Remote Deployment Process - Java.
- Started by the "deployment.yml" playbook.
- Posts progress messages/events to Event Logger web service.
Shell Scripts
dbbackup.sh
- Uses RMAN to take a full backup of the database on remote servers
- Called by "deployment.jar".
- Executed before applying updates to the database on remote servers.
Where can I find all of this stuff?
Remote Connections Application
- http://sourcecontrol.fullcount.net/repos/internal/trunk/at-remote-connection
- sftp://root@fcadmin01.fullcount.net/srv/tomcat/webapps/at-remote-connection
- http://sourcecontrol.fullcount.net/repos/packages/fullcount/server/base/trunk/src/usr/bin
- http://sourcecontrol.fullcount.net/repos/packages/fullcount/server/base/trunk/src/etc/cron.d
- http://sourcecontrol.fullcount.net/repos/server/central/trunk/remote_connections/scripts/home/tunnel/port-range-finder
- sftp://root@fcadmin01.fullcount.net/home/tunnel/port-range-finder
SSH Keys and Authentication Agent
- http://sourcecontrol.fullcount.net/repos/packages/fullcount/server/base/trunk/src/root/.ssh/authorized_keys
- sftp://root@fcadmin01.fullcount.net/home/tunnel/.ssh/authorized_keys
- http://sourcecontrol.fullcount.net/repos/passwords/SSH Keys.kdbx
- J:\FullCount\Passwords\SSH Keys.kdbx
Ansible Inventory Script
- http://sourcecontrol.fullcount.net/repos/ansible/trunk/inventory/inventory.py
- sftp://root@fcadmin01.fullcount.net/srv/ansible/inventory/inventory.py
Ansible Playbooks
Shell Scripts
Deployment Tool
How do I actually do a deployment?
See How to Use Ansible to Deploy Applications on Remote Servers.
Related Articles
Run Remote Deployment Process - Java
Remote Server Configuration This has already been completed for all existing remote servers. Java 1.8 installed Tomcat located at /srv/tomcat XE database installed and running with fc_obj_owner user Directory /home/deployment exists Database backup ...Remote Deployment Process - Java
Deployment Process Determine if a valid argument has been passed in Only continue if downloadOnly, postDownload, fullDeployment, handleSuccess, checkHash, verifyApplications passed through. Lock file to stop deployment from running more than once ...Copy-down Process for Remote Servers
Copy-down Process for Remote Servers Note: This is just the "copy-down" process; it does not include steps for installing or configuring the OS, database, or applications. Create a new directory on "fcadmin01.fullcount.net" with the community code ...Remote Server Decommission Process
Remote Server Decommission Process The purpose of this document is to track the process for decommissioning a FullCount remote server as part of the transition for the client to be centrally hosted or if the client is leaving FullCount. The first ...How to Use Ansible to Deploy Applications on Remote Servers
How to Use Ansible to Deploy Applications on Remote Servers Objectives Use ansible to deploy applications on remote servers. Prerequisites A VPN account. A VPN client installed on your computer. An ssh client installed on your computer. An ssh ...