Tod is for: Template Open Deploy. It is tailored around docker-compose on Linux, taking advange of git for tracking changes to the runtime environment of Docker containers.
Tod allows one to manage one or more Docker container by tracking its or their runtime enviroment. Enviroment variables, configuration files and other kind of resources that containers make use of at runtime are represented inside tod, so they are available and version controlled in one place and ready to be deployed.
Tod can generate a new enviroment, make changes and integrate them inside an existing one, sometime without even needing to restart containers.
To try out and get a feel of what tod does, run:
MODALL=1 ./tod --check=test1 MODALL=1 ./tod --doins=test1,test2
MODALL is to force tod to use also blacklisted modules, since test1 and test2 are indeed blacklisted by default. Check --help for more details.
Tod is designed for rather small sized, non professional projects. One of its goals is to ease deployment not development.
Development should happen independently, tod can work with external packages (such as dpkg, apt-get, pip) and sources (like git or tarballs), they can be added to the tod Manifest file and imported on a per module basis.
The idea behind tod is to gather and centralise resources, making them available programmatically in order to automate the process of deploying contents and services on a public domain. Tod can work with binaries, sources and contents of various kind.
Tod uses the concept of Module for representing an independent set of files that can depend on external resources like binaries, sources or other modules.
Every module must have at least three files: module.sh, holder.sh and scheme.sh.
They are described in their own section respectively, for now worth saying that module.sh accounts for the logic necessary for configuring a template and external resources like packages and sources, holder.sh is a very stripped down version of a bash script, its job is to define the value of placeholder variables used directly in templates. scheme.sh is a collection of templates, or in other words, prospective files that tod can output and dump in a dedicated location.
Once the whole runtime environment has been represented through one or more tod modules, tod can generate one instance of environment that docker-compose will actually make use of.
Once an instance has been generated should be possible to use plain Docker to build images and run containers pointing them to the tod output location. That is to say, from a Docker point of view tod doesn't exist.
Tod output location is referred to as instance_d. Its tree structure is illustrated below. Tod can generate new almost identical instances, almost since one instance could rely on external packages that aren't directly tracked within tod itself. This is how an instance_d working tree looks in tod:
instance_d/
``````````` docker-compose.yml
` docker/ <------- Docker build-time files
````````` app_a/
` ```````` Dockerfile
` ` ` ...
` ` ` app.env
` ` ...
` ` app_z/
` ```````` Dockerfile
` ` ...
` ` entrypoint.sh
` rootfs/ <------- Docker run-time files
````````` bin/
` ...
` var/
Under instance_d/docker there are Docker build time configuration files. instance_d/rootfs contains configuration files, directories and resources that containers need at runtime environment. Beware that it is recommended to install in rootfs everything that is meant to be mounted into Docker containers. It is not recommended to establish mappings toward paths outside the rootfs of an instance. In case rootfs needs resources located on the host machine itself, it is possible to symlink from it to the external host file system.
Last, but not least, tod offers some validation and automatic resolving of dependency features. The idea is that modules are isolated, so they don't need to worry for namespaces, don't need to explicitly source other modules that could depend on. Tod can work out name clashes and dependencies, give warnings and errors that will - hopefully - help in keeping the overall state of the runtime Docker instance consistent.
A module is a set of valid bash script files that depend on each other. They must live all together inside the same directory. This directory must live under $module_d/ and its name must match with - and de facto it is - the name of the module. For instance modules called test1 and test2 will be accessible at $module_d/test1 and $module_d/test2. Every module must at least define three files: holder.sh, module.sh and scheme.sh. To visualise modules arrangement see below:
$tod/config
-----+-----
|
$module_d/common.sh (inherited from all modules)
---------+---------
|
$module_d/mod_a $module_d/mod_b | $module_d/mod_c
-------+-----------------+---------+---------+------------- ...
| | |
\ module.sh \ module.sh |
\ holder.sh \ holder.sh <--+ |
\ scheme.sh \ scheme.sh | \ module.sh
+--- \ holder.sh
(dep) \ scheme.sh
module.sh is the main file, module.sh depends on config and common.sh. It cannot have external dependencies other than these two files. It can implement a common interface to expose internal services to tod that can then call back into every module.sh without knowing their own implementation details. These are the callbacks that every module.sh can implement:
module.sh is where every module can define its own logic to fetch, configure and build (if necessary) external resources. The final result must be copied into instance_d - in a make install fashion - where it will be available for Docker.
holder.sh must follow a special syntax in order to pass the validation layer. If it doesn't pass, then the process exits with an error. Syntax restrictions are described below. Consider holder.sh as a list of labels that tod will stick onto every template at their creation time.
holder.sh always depends on the module.sh within the same module and can depend on other modules holder.sh. holder.sh doesn't need to include dependencies by any means or explicitly, tod can work them out behind the scenes. In case tod cannot resolve one dependency, then it stops and holder.sh needs fixing.
holder.sh is the only module file that can have external dependencies, which is to say, it can depend on other modules holder.sh. Conceptually it can define a variable in terms of another holder.sh variable.
It is possible to give each template defined in scheme.sh a file access mode by defining related *_UID, *_GID and *_FMODE variables in the holder.sh script. For instance, suppose a module called testmode defines a template named my_conf_t. It is possible to specify different file mode attributes this file needs to have by defining the following variables in testmode/holder.sh:
_TESTMOD_MY_CONF_T_FMODE_="0640" _TESTMOD_MY_CONF_T_GID_="33" _TESTMOD_MY_CONF_T_UID_="33"
Those variables are independent from each other, special care in their naming must be taking in order for io.sh to be able to pick them up when performing file I/O. The name structure for those special variables is as follows:
_<capital module name>_<capital template name>_<{UID|GID|FMODE}>_
scheme.sh must depend on its own module's holder.sh only. It defines one or more templates to be expanded and written to file, targeting those files that Docker or docker-compose may need for containers build or runtime.
The idea is that, once instance_d is ready, then Docker or docker-compose do not care about anything that tod did, they can be invoked being totally unaware of how instance_d was generated, so working with Docker is completely decoupled from tod.
Every module must pass bash set -o errexit, this is the first requirement, otherwise the module is excluded and cannot be used. Any other module that depends on a broken module is also excluded, normally causing the whole process to exit with an error.
holder.sh can only use the following subset of legal bash syntactic constructs:
#if / else / elif / fi statement=+=$()These constructs are only allowed in the following form:
There must be only one assignment per line, no special keywords such as declare, readonly or any other, no line breaks \. Each assignment must fit into one and only one line. The lhs of the assignment is checked against the regular expression $rex_legal_holder_assignment_lhs. Such regex enforces a pattern for holder variables naming that must always be met to pass validation. Rhs is checked against $rex_illegal_holder_assignment_rhs, this enforces nothing to share the same line together with the assignment itself.
A valid variable name must:
__Here is some example of valid variable names, assuming holder.sh is part of a module called testmod:
# Valid LHS variable names: _TESTMOD_HOLDERVAR1_ _TESTMOD_2HOLDERVAR_ _TESTMOD_THREE_TOKENSLHS_
A valid assigned value must be always enclosed between double quotes, there is no restriction whatsoever on characters and special symbols for the assigned value. Extending the examples above to also include the right hand side of the assignment:
# Valid assignments: _TESTMOD_HOLDERVAR1_="@#!=+{This can be \"whatever\"}+=!#@" _TESTMOD_2H0LD3RV4R_="1" _TESTMOD_THREE_TOKENSLHS_="example_of_rhs"
Check tod/module/test1 and tod/module/test2 for further reference.