General #
Mailborder servers report, make changes, update configurations, and perform other procedures via a task system. By default, the task scheduler runs every minute to check for new tasks to perform. This is controlled via the system cron scheduler.
Concept #
Due to security concerns, changes to Mailborder servers cannot be made directly from the GUI. These changes often require root access to the server, so changes are performed by putting a task request into the database. Every minute Mailborder servers check for any new tasks. When one is found, it is then run by a command line executable. The database is then updated to reflect the status of the task. (Pending, Running, Complete, Aborted)
Master server command line tools can access the database directly via the database service running on the loopback interface. (127.0.0.1) Child servers cannot access the database, so they communicate with the Master through the Master server’s API. These Child server connections are strictly managed by the API eliminating the possibility of any type of code injections. The communications are secured through mandatory TLS connections, hashes, and structured requests.
Configuration Changes #
When a change is made in the Master GUI that requires a configuration change, a task is written to the database for the Master server to update the cluster’s criterion database. The criterion database is an SQLite database that most Mailborder executable files access to rebuild their configurations and rulesets. This database is created by the Master server and then distributed to the Child servers. This database is a very thin version of the Master server database that contains only configuration options stored on the Master server. There are no logs, accounts, passwords, or other type of sensitive data.
Once the Master server completes the task of creating a new criterion database, it then creates a task for itself to rebuild its configuration files. This same task is also created for any Child servers within the cluster. Upon the next task check controlled by the system cron, the Master server will execute the rebuild task to update the running configuration. When Child servers contact the Master looking for new tasks, a rebuild task also signals the Child server to download the new criterion database from the Master server before it rebuilds its configurations.
Since by default there is up to a one minute delay in the cron scheduler for tasks, changes can take up to two minutes to complete. The first for the criterion rebuild, and the second for the configuration rebuild.
Stuck Tasks #
If there is an interruption in communication, it is possible that a task be left as Running in the database. For example, if a Child server checks for a rebuild task and one is waiting, the Child server will notify the Master server that the task is acknowledged and its status is changed from Pending to Running. Once the task is complete, the Child server will notify the Master server is complete and the task is marked as Completed.
However, if something happens when the task is running, such as a system reboot, the Master server will not be notified the task is complete. This will leave that task in the Running stage. Since the database is checked for any Pending or Running tasks before scheduling a new task, this stuck task could leave the target server in a state where it will no longer receive new rebuild tasks.
The Master server monitors for Running tasks. If a task has been marked as Running for too long, it will automatically abort the task and create a new one as required. An administrator also has the option to Abort a task in the GUI from the Task Log page.
So, if you notice a server in your cluster is not getting any new rules, check the Task Log page in the Master GUI for any tasks stuck in the Running state that are old. For example, if a task is stuck in Running and is three hours old, that task is stuck. The server that was assigned that task has probably already completed the task but was unable to notify the Master server of the new status.
