Introduce tod - Template Open Deploy
diff --git a/MANIFEST b/MANIFEST
new file mode 100644
index 0000000..7d93cc6
--- /dev/null
+++ b/MANIFEST
@@ -0,0 +1,35 @@
+#!/bin/bash
+#
+# MANIFEST
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# @@MIRROR_BEGIN@@
+#module=
+#mirror=
+#protocol=
+#branch=
+#host=
+#user=
+#format=
+#file=
+#method={git-archive|git-clone|wget-file}
+# @@MIRROR_ENDED@@
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..88acbad
--- /dev/null
+++ b/README.md
@@ -0,0 +1,264 @@
+# Tod
+
+## Template Open Deploy - Overview
+----------------------------------
+
+Tod is for: Template Open Deploy. It is tailored around docker-compose on Linux,
+taking advange of git for tracking changes to the runtime environment of Docker
+containers.
+
+Tod allows one to manage one or more Docker container by tracking its or their
+runtime enviroment. Enviroment variables, configuration files and other kind of
+resources that containers make use of at runtime are represented inside tod, so
+they are available and version controlled in one place and ready to be deployed.
+
+Tod can generate a new enviroment, make changes and integrate them inside an
+existing one, sometime without even needing to restart containers.
+
+To try out and get a feel of what tod does, run:
+
+```
+MODALL=1 ./tod --check=test1
+MODALL=1 ./tod --doins=test1,test2
+```
+
+MODALL is to force tod to use also blacklisted modules, since test1 and test2
+are indeed blacklisted by default. Check --help for more details.
+
+Tod is designed for rather small sized, non professional projects. One of its
+goals is to ease *deployment not development*.
+
+Development should happen independently, tod can work with external packages
+(such as dpkg, apt-get, pip) and sources (like git or tarballs), they can be
+added to the tod Manifest file and imported on a per module basis.
+
+### End goal
+------------
+
+The idea behind tod is to gather and centralise resources, making them available
+programmatically in order to automate the process of deploying contents and
+services on a public domain. Tod can work with binaries, sources and contents of
+various kind.
+
+### Instance
+------------
+
+Tod uses the concept of *Module* for representing an independent set of files
+that *can* depend on external resources like binaries, sources or other modules.
+
+Every module must have at least three files: module.sh, holder.sh and scheme.sh.
+
+They are described in their own section respectively, for now worth saying that
+*module.sh* accounts for the logic necessary for configuring a template and
+external resources like packages and sources, *holder.sh* is a very stripped
+down version of a bash script, its job is to define the value of *placeholder
+variables* used directly in templates. *scheme.sh* is a collection of templates,
+or in other words, prospective files that tod can output and dump in a dedicated
+location.
+
+Once the whole runtime environment has been represented through one or more tod
+modules, tod can generate one instance of environment that docker-compose will
+actually make use of.
+
+Once an instance has been generated should be possible to use plain Docker to
+build images and run containers pointing them to the tod output location. That
+is to say, from a Docker point of view tod doesn't exist.
+
+Tod output location is referred to as `instance_d`. Its tree structure is
+illustrated below. Tod can generate new _almost_ identical instances, _almost_
+since one instance could rely on external packages that aren't directly tracked
+within tod itself. This is how an `instance_d` working tree looks in tod:
+
+```
+ instance_d/
+ ``````````` docker-compose.yml
+ ` docker/ <------- Docker build-time files
+ ````````` app_a/
+ ` ```````` Dockerfile
+ ` ` ` ...
+ ` ` ` app.env
+ ` ` ...
+ ` ` app_z/
+ ` ```````` Dockerfile
+ ` ` ...
+ ` ` entrypoint.sh
+ ` rootfs/ <------- Docker run-time files
+ ````````` bin/
+ ` ...
+ ` var/
+```
+
+Under `instance_d/docker` there are Docker build time configuration files.
+`instance_d/rootfs` contains configuration files, directories and resources that
+containers need at runtime environment. Beware that it is recommended to install
+in `rootfs` everything that is meant to be mounted into Docker containers. It
+is not recommended to establish mappings toward paths outside the rootfs of an
+instance. In case `rootfs` needs resources located on the host machine itself,
+it is possible to symlink from it to the external host file system.
+
+## Validation
+-------------
+
+Last, but not least, tod offers some validation and automatic resolving of
+dependency features. The idea is that modules are isolated, so they don't need
+to worry for namespaces, don't need to explicitly source other modules that
+could depend on. Tod can work out name clashes and dependencies, give warnings
+and errors that will - _hopefully_ - help in keeping the overall state of the
+runtime Docker instance consistent.
+
+## Modules
+----------
+
+A *module* is a set of valid bash script files that depend on each other. They
+must live all together inside the same directory. This directory must live under
+`$module_d/` and its name must match with - and de facto it is - the name of the
+module. For instance modules called `test1` and `test2` will be accessible at
+`$module_d/test1` and `$module_d/test2`. Every module must at least define three
+files: holder.sh, module.sh and scheme.sh. To visualise modules arrangement see
+below:
+
+```
+ $tod/config
+ -----+-----
+ |
+ $module_d/common.sh (inherited from all modules)
+ ---------+---------
+ |
+ $module_d/mod_a $module_d/mod_b | $module_d/mod_c
+ -------+-----------------+---------+---------+------------- ...
+ | | |
+ \ module.sh \ module.sh |
+ \ holder.sh \ holder.sh <--+ |
+ \ scheme.sh \ scheme.sh | \ module.sh
+ +--- \ holder.sh
+ (dep) \ scheme.sh
+```
+
+### module.sh
+-------------
+
+module.sh is the main file, module.sh depends on `config` and `common.sh`. It
+cannot have external dependencies other than these two files. It can implement a
+common interface to expose internal services to tod that can then call back into
+every module.sh without knowing their own implementation details. These are the
+callbacks that every module.sh *can* implement:
+
+0. tod_check
+1. tod_watch
+2. tod_fetch
+3. tod_doall
+4. tod_upall
+5. tod_doins
+6. tod_upins
+7. tod_clins
+8. tod_upmod
+9. tod_clmod
+
+module.sh is where every module can define its own logic to fetch, configure and
+build (if necessary) external resources. The final result must be copied into
+`instance_d` - in a _make install_ fashion - where it will be available for
+Docker.
+
+### holder.sh
+-------------
+
+holder.sh must follow a special syntax in order to pass the validation layer. If
+it doesn't pass, then the process exits with an error. Syntax restrictions are
+described below. Consider holder.sh as a list of labels that tod will stick onto
+every template at their creation time.
+
+holder.sh always depends on the module.sh within the same module and can depend
+on other modules holder.sh. holder.sh doesn't need to include dependencies by
+any means or explicitly, tod can work them out behind the scenes. In case tod
+cannot resolve one dependency, then it stops and holder.sh needs fixing.
+
+holder.sh is the only module file that can have external dependencies, which is
+to say, it can depend on other modules holder.sh. Conceptually it can define a
+variable in terms of another holder.sh variable.
+
+### scheme.sh
+-------------
+
+scheme.sh must depend on its own module's holder.sh only. It defines one or more
+templates to be expanded and written to file, targeting those files that Docker
+or docker-compose may need for containers build or runtime.
+
+The idea is that, once `instance_d` is ready, then Docker or docker-compose do
+not care about anything that tod did, they can be invoked being totally unaware
+of how `instance_d` was generated, so working with Docker is completely
+decoupled from tod.
+
+#### Bash
+
+Every module must pass `bash set -o errexit`, this is the first requirement,
+otherwise the module is excluded and cannot be used. Any other module that
+depends on a broken module is also excluded, normally causing the whole process
+to exit with an error.
+
+#### Syntax
+
+holder.sh can only use the following subset of legal bash syntactic constructs:
+
+1. blank lines
+2. comment lines `#`
+3. `if` / `else` / `elif` / `fi` statement
+4. variable assignment `=`
+5. variable incremental assignment `+=`
+6. command substitution `$()`
+
+These constructs are only allowed in the following form:
+
+* for (1) no restrictions, any valid bash blank like is also valid.
+* for (2) no restrictions, any valid bash comment is also valid.
+* for (3) if, else, elif, fi keywords are only valid as first word in a line.
+* for (4) and (5) variable assignment is only valid in a special form, see below
+ the 'Variable assignment' section.
+* for (6) no restrictions, any valid bash command substitution.
+
+#### Variable assignment
+
+There must be only one assignment per line, no special keywords such as
+`declare`, `readonly` or any other, no line breaks `\`. Each assignment must fit
+into one and only one line. The lhs of the assignment is checked against the
+regular expression `$rex_legal_holder_assignment_lhs`. Such regex enforces a
+pattern for holder variables naming that must always be met to pass validation.
+Rhs is checked against `$rex_illegal_holder_assignment_rhs`, this enforces
+nothing to share the same line together with the assignment itself.
+
+#### Variable name - LHS
+
+A valid variable name must:
+
+1. Be composed of at least two tokens
+2. Have each token starting by an underscore, `_`
+3. Have each token in upper case English alphabet and/or numbers, A-Z, 0-9
+4. Have the first token named after the module where it belongs to
+5. End by an underscore, `_`
+
+Here is some example of valid variable names, assuming holder.sh is part of a
+module called `testmod`:
+
+```bash
+# Valid LHS variable names:
+
+_TESTMOD_HOLDERVAR1_
+_TESTMOD_2HOLDERVAR_
+_TESTMOD_THREE_TOKENSLHS_
+```
+
+#### Variable value - RHS
+
+A valid assigned value must be always enclosed between double quotes, there is
+no restriction whatsoever on characters and special symbols for the assigned
+value. Extending the examples above to also include the right hand side of the
+assignment:
+
+```bash
+# Valid assignments:
+
+_TESTMOD_HOLDERVAR1_="@#!=+{This can be \"whatever\"}+=!#@"
+_TESTMOD_2H0LD3RV4R_="1"
+_TESTMOD_THREE_TOKENSLHS_="example_of_rhs"
+```
+
+Check `tod/module/test1` and `tod/module/test2` for further reference.
diff --git a/config b/config
new file mode 100644
index 0000000..8127a4b
--- /dev/null
+++ b/config
@@ -0,0 +1,159 @@
+#!/bin/bash
+#
+# config - for tod
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# Absolute path to where this script lives
+declare -rg script_dir="$(get_script_dir)"
+
+# Runtime timestamp of when this script was executed
+declare -rg script_now="$(get_script_now)"
+
+# Version file name
+declare -rg __version="version"
+
+# Module staging area name
+declare -rg staging="staging"
+
+# Name of this host's instance, it defaults to localhost
+declare -rg def_hostname="localhost"
+declare -rg host_name="${HOST_NAME:-$def_hostname}"
+
+# Switch for enabling debug features, it defaults to debug
+declare -rg release="release"
+declare -rg debug="debug"
+declare -rg instance_mode="${INSTANCE_MODE:-$debug}"
+
+# Force to use also blacklisted modules
+declare -rgi modall="${MODALL:-0}"
+
+# Default answer instead of waiting for user input
+declare -rg answer=${DEFANS:-}
+
+# Runtime enabled list of modules
+declare -ag MODULES=( )
+
+# Blacklist modules
+declare -arg BLMODULES=( "test1" "test2" )
+
+# Runtime running option
+declare -Ag OPTIONS=( \
+ [check]=0 [watch]=0 [fetch]=0 [doall]=0 [doins]=0 \
+ [upins]=0 [upmod]=0 [upall]=0 [clmod]=0 [clins]=0 \
+)
+
+# Instance tree
+declare -rg def_instance_d="$script_dir/$host_name"
+declare -rg instance_d="${INSTANCE_DIR:-$def_instance_d}"
+declare -rg docker_d="$instance_d/docker"
+declare -rg rootfs_d="$instance_d/rootfs"
+
+# Module tree
+declare -rg module_d="$script_dir/module"
+declare -rg scheme_sh="scheme.sh"
+declare -rg holder_sh="holder.sh"
+declare -rg module_sh="module.sh"
+
+# Runtime password file
+declare -rg passwd_sh="${PASSWD_F:-}"
+
+# Module common files
+declare -rg common_sh="$module_d/common.sh"
+declare -rg manifest_f="$script_dir/MANIFEST"
+declare -rg uninstall_f="$instance_d/uninstall.sh"
+
+# Set of regular expressions for accomplishing module validation
+declare -rg rex_legal_holder_token="_([0-9]|[A-Z])+"
+declare -rg rex_legal_holder_variable="($rex_legal_holder_token)+_"
+declare -rg rex_legal_holder_assignment_op="[\+]?="
+declare -rg rex_legal_holder_assignment_lhs=\
+"$rex_legal_holder_variable$rex_legal_holder_assignment_op\""
+declare -rg rex_illegal_holder_assignment_rhs=".+[^\\]\".+"
+declare -rg dynamic_typedef="local -r"
+declare -rg rex_legal_dynamic_typedef="local[[:space:]]\-r"
+declare -rg rex_legal_dynamic_assignment_lhs=\
+"$rex_legal_dynamic_typedef $rex_legal_holder_assignment_lhs"
+declare -rg head_tag=">>>>>"
+declare -rg foot_tag="<<<<<"
+declare -rg rex_legal_file_header_tag="#[[:space:]]$head_tag[[:space:]].*"
+declare -rg rex_legal_file_footer_tag="#[[:space:]]$foot_tag[[:space:]].*"
+declare -rg rex_legal_manifest_hook_begin="@@MIRROR_BEGIN@@"
+declare -rg rex_legal_manifest_hook_ended="@@MIRROR_ENDED@@"
+
+# Credits
+declare -rg credits_year="2019"
+declare -rg credits_author="Luigi Santivetti"
+declare -rg credits_email="luigi.santivetti@gmail.com"
+declare -rg credits_vers="1.0 beta"
+
+declare -rg license="\
+# Copyright ${credits_year} ${credits_author} <${credits_email}>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the \"Software\"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# This file is autogenerated with tod ${credits_vers} and manual edit can break
+# your instance. Update this file using tod instead, check tod --help."
+
+# Logging and debug
+declare -irg LOG_D="${LOG_D:-0}"
+declare -irg LOG_I="${LOG_I:-1}"
+declare -irg LOG_W="${LOG_W:-1}"
+declare -irg LOG_X="${LOG_X:-1}"
+
+# Return codes
+declare -ir s_ok=0
+declare -ir s_err=1
+declare -ir s_unbound=2
+declare -ir s_maybe_unbound=3
+declare -ir s_bound=4
+declare -ir s_null=5
+declare -ir s_unk_stdout=6
+declare -ir s_unk_stderr=7
+declare -ir s_inv_lhs=8
+declare -ir s_err2=9
+declare -ir s_err3=10
+declare -ir s_err4=11
+declare -ir s_err5=12
+declare -ir s_err6=13
+declare -ir s_err7=14
+declare -ir s_err8=15
+declare -ir s_err9=16
+declare -ir s_disabled=17
+declare -ir s_off_broken=18
+declare -ir s_off_usable=19
+declare -ir s_maybe_bound=20
diff --git a/module/common.sh b/module/common.sh
new file mode 100644
index 0000000..4b6e4bd
--- /dev/null
+++ b/module/common.sh
@@ -0,0 +1,135 @@
+#!/bin/bash
+#
+# common - module.sh
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+declare -r mod_module_d="$module_d/$module"
+declare -r mod_instance_d="$instance_d"
+declare -r mod_docker_d="$docker_d/$module"
+declare -r mod_rootfs_d="$rootfs_d"
+declare -r mod_mode_env="${module^^}_MODE"
+declare -r mod_mode="$(eval "echo \${$mod_mode_env:-$instance_mode}")"
+declare -r mod_staging_d="$mod_module_d/$staging"
+
+declare -ar mod_dirs=( $mod_instance_d $mod_docker_d $mod_rootfs_d )
+declare -ar mod_files=( )
+declare -ar mod_trefs=( )
+
+if [ "${mod_mode}" != "${debug}" ] && [ "${mod_mode}" != "${release}" ]; then
+ exit 1
+fi
+
+# @1 : module name
+# @2 : absolute path to module staging dir
+function __fetch_module_common
+{
+ local modname
+ local tmp_d
+ local tar
+
+ [ -z "$1" ] && {
+ [ -n "$module" ] || {
+ lets -l -e "invalid module"; return $s_err
+ }; modname="$module"
+ } || modname=$1
+
+ [ -z "$2" ] && {
+ [ -n "$mod_staging_d" ] && [ -d "$mod_module_d" ] || {
+ lets -l -e "invalid module root"; return $s_err
+ }; tmp_d="$mod_staging_d"
+ } || tmp_d=$2
+
+ is_unix_path $tmp_d || { lets -l -e "invalid staging area"; return $s_err; }
+
+ [ -d "$tmp_d" ] && {
+ lets --ask "${tmp_d##$module_d/} existing, wipe it?" && rm -rf $tmp_d || \
+ lets -l -w "keeping staging"
+ }
+
+ mkdir -p $tmp_d && process_manifest $modname $tmp_d || {
+
+ # Wipe the whole staging_d for this module
+ [ "$tmp_d" = "$mod_staging_d" ] && rm -rf $tmp_d
+ return $s_err
+ }
+}
+
+function __dir_do_common_helper
+{
+ local -a __a1=( )
+
+ if [ -z "${mod_more_dirs[*]}" ] || [ ${#mod_more_dirs[@]} -eq 0 ]; then
+ lets -l -w "$module: does not have more dirs"
+ __a1=( "${mod_dirs[@]}" )
+ else
+ __a1=( $(__merge_array "${mod_dirs[@]}" "${mod_more_dirs[@]}") )
+ fi
+
+ create_directory "${__a1[@]}" || return $s_err
+}
+
+function __file_do_common_helper
+{
+ local -a __a1=( ); local -a __a2=( )
+
+ if [ -z "${mod_more_files[*]}" ] || [ ${#mod_more_files[@]} -eq 0 ]; then
+ lets -l -w "$module: does not have more files"
+ else
+ __a1=( $(__merge_array "${mod_files[@]}" "${mod_more_files[@]}") )
+ fi
+
+ if [ -z "${mod_more_trefs[*]}" ] || [ ${#mod_more_trefs[@]} -eq 0 ]; then
+ lets -l -w "$module: does not have more templates"
+ else
+ __a2=( $(__merge_array "${mod_trefs[@]}" "${mod_more_trefs[@]}") )
+ fi
+
+ __file_do "$1" "$uninstall_f" "${__a1[@]}" "${__a2[@]}" || return $s_err
+}
+
+function __doins_module_common
+{
+ __dir_do_common_helper && __file_do_common_helper --create
+}
+
+function __watch_module_common
+{
+ __file_do_common_helper --splash
+}
+
+function __upins_module_common
+{
+ __dir_do_common_helper && __file_do_common_helper --update
+}
+
+function __clmod_module_common
+{
+ rm -rf $mod_staging_d || sudo rm -rf $mod_staging_d
+}
+
+function __clins_module_common
+{
+ local -a __a1=( ); local -a __a2=( )
+
+ __file_do --update \
+ "$uninstall_f" "${__a1[@]}" "${__a2[@]}" || return $s_err
+}
diff --git a/module/test1/holder.sh b/module/test1/holder.sh
new file mode 100644
index 0000000..5a868e4
--- /dev/null
+++ b/module/test1/holder.sh
@@ -0,0 +1,89 @@
+#!/bin/bash
+#
+# holder.sh - test
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# Test plain variable
+_TEST1_VAR0_HOLDER_="test1_var0_holder"
+
+# Test variable externally defined within test1 module
+_TEST1_VAR1_MODULE_="$mod_name_clash"
+
+# Test variable externally defined in foreign holder and composite
+_TEST1_VAR2_EXTREF_="${_TEST2_VAR0_HOLDER_}-notnull"
+
+# Test variable externally defined in foreign module and composite
+_TEST1_VAR3_EXTREF_="${_TEST2_VAR1_MODULE_}-notnull"
+
+# Test circular dependencies. Uncomment will cause validation to detect a loop
+#_TEST1_VAR4_EXTREF_="${_TEST2_VAR4_HOLDER_}"
+#_TEST1_VAR5_EXTREF_="${_TEST2_VAR5_MODULE_}"
+
+# Test variable externally defined in foreign holder
+_TEST1_VAR6_EXTREF_="${_TEST2_VAR0_HOLDER_}"
+
+# Test variable externally defined in foreign module
+_TEST1_VAR7_EXTREF_="${_TEST2_VAR1_MODULE_}"
+
+# Test command substitution with unescaped quote symbols
+_TEST1_VAR8_CMDSUB_="text $(dirname "$(basename "$(dirname "$(pwd)")")")"
+
+# Test LHS validation. Uncomment will cause validation to detect invalid LHS
+#_NOVALID_VAR9_LHS_="invalid LHS. First token must match module's name"
+
+# Test RHS validation. Uncomment will cause validation to detect multiple
+# commands in line.
+#_TEST1_VAR10_MULTIPLE_CMD_="legal RHS"; echo "Illegal continuation"
+#_TEST1_VAR11_MULTIPLE_CMD_="legal RHS" && _TEST1_VAR12_="illegal continuation"
+#_TEST1_VAR13_MULTIPLE_CMD_="legal RHS"; if :;then echo "true"; fi
+
+# Test if/else
+if true; then
+ _TEST1_VAR14_="true"
+else
+ _TEST1_VAR15_="false"
+fi
+
+# Test incremental assignment
+_TEST1_VAR16_="var15"
+_TEST1_VAR16_+=" + var15"
+
+# FIXME: test for null value. Validation does not pick this up if in the same
+# module. holder.sh will never trigger an error cause RHS isn't checked for
+# local holder variables.
+_TEST1_VAR17_=""
+
+# Test for reference null value. Uncomment will cause validation to detect a null
+# value for a referenced variable.
+#_TEST1_VAR17_="${_TEST2_VAR6_EXTREF_}"
+
+# Test for reference unbound. Uncomment will cause validation to detect an unbound
+# variable.
+#_TEST1_VAR18_="${_TEST2_VAR6_EXTREF_}"
+
+# Test for unknown variable name. Uncomment will cause validation to detect an
+# unknown variable name on the RHS.
+#_TEST1_VAR19_="${this_isn_t_a_valid_reference_name}"
+
+# Test for invalid module name. Uncomment will cause validation to detect an
+# invalid module name.
+#_TEST1_VAR20_="${_MODULEFOO_DOES_NOT_EXISTS_}"
diff --git a/module/test1/module.sh b/module/test1/module.sh
new file mode 100644
index 0000000..00e4360
--- /dev/null
+++ b/module/test1/module.sh
@@ -0,0 +1,64 @@
+#!/bin/bash
+#
+# module.sh - test1
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+declare -r module="test1"
+source $common_sh
+
+module_enable $module
+
+declare -r tmp_d="${mod_rootfs_d}/tmp"
+declare -r test="${module}-testfile.txt"
+declare -r test_f="$tmp_d/$test"
+
+declare -r mod_name_clash="test1_var1_module"
+
+declare -ar mod_more_dirs=( $tmp_d )
+declare -ar mod_more_files=( $test_f )
+declare -ar mod_more_trefs=( test_t )
+
+# Test file removal. Tod should be able to uninstall files by looking at what
+# appears in $instance_d/uninstall.sh. In this case if $test_f was installed,
+# then by uncommenting these lines would exclude it and cause tod to remove it.
+#declare -ar mod_more_files=( )
+#declare -ar mod_more_trefs=( )
+
+function tod_watch
+{
+ __watch_module_common || return $s_err
+}
+
+function tod_doins
+{
+ __doins_module_common || return $s_err
+}
+
+function tod_upins
+{
+ __upins_module_common || return $s_err
+}
+
+function tod_clins
+{
+ __clins_module_common || return $s_err
+}
diff --git a/module/test1/scheme.sh b/module/test1/scheme.sh
new file mode 100644
index 0000000..1801fdb
--- /dev/null
+++ b/module/test1/scheme.sh
@@ -0,0 +1,40 @@
+#!/bin/bash
+#
+# holder.sh - test
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+declare -r test_t="\
+
+test1 - scheme.sh - test_t
+
+@@ START >>>>>>>>>>>>>
+${_TEST1_VAR0_HOLDER_}
+${_TEST1_VAR1_MODULE_}
+${_TEST1_VAR2_EXTREF_}
+${_TEST1_VAR3_EXTREF_}
+\${_TEST1_VAR4_EXTREF_}
+\${_TEST1_VAR5_EXTREF_}
+${_TEST1_VAR6_EXTREF_}
+${_TEST1_VAR7_EXTREF_}
+\${_TEST3_VAR7_UNBOUND_}
+@@ END <<<<<<<<<<<<<<<
+"
diff --git a/module/test2/holder.sh b/module/test2/holder.sh
new file mode 100644
index 0000000..747bd97
--- /dev/null
+++ b/module/test2/holder.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+#
+# holder.sh - test
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+_TEST2_VAR0_HOLDER_="test2_var0_holder"
+_TEST2_VAR1_MODULE_="$mod_name_clash"
+#_TEST2_VAR2_EXTREF_="${_TEST1_VAR0_HOLDER_}-notnull"
+#_TEST2_VAR3_EXTREF_="${_TEST1_VAR1_MODULE_}-notnull"
+
+# Test circular dependencies. Uncomment will cause validation to detect a loop
+#_TEST2_VAR4_EXTREF_="${_TEST1_VAR0_HOLDER_}"
+#_TEST2_VAR5_EXTREF_="${_TEST1_VAR1_MODULE_}"
+
+# Test for reference with null value. Uncomment will cause validation to detect
+# a null value. Which is not allowed.
+#_TEST2_VAR6_EXTREF_=""
diff --git a/module/test2/module.sh b/module/test2/module.sh
new file mode 100644
index 0000000..9e0ec5e
--- /dev/null
+++ b/module/test2/module.sh
@@ -0,0 +1,61 @@
+#!/bin/bash
+#
+# module.sh - test2
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+declare -r module="test2"
+source $common_sh
+
+# Test dependencies. Comment will cause validation to detect that test1 has one
+# unmet dependency on test2.
+module_enable $module
+
+declare -r tmp_d="${mod_rootfs_d}/tmp"
+declare -r test="${module}-testfile.txt"
+declare -r test_f="$tmp_d/$test"
+
+# Test name clash. test1 and test2 both declare a global variable mod_name_clash
+declare -r mod_name_clash="test2_var1_module"
+
+declare -ar mod_more_dirs=( $tmp_d )
+declare -ar mod_more_files=( $test_f )
+declare -ar mod_more_trefs=( test_t )
+
+function tod_watch
+{
+ __watch_module_common || return $s_err
+}
+
+function tod_doins
+{
+ __doins_module_common || return $s_err
+}
+
+function tod_upins
+{
+ __upins_module_common || return $s_err
+}
+
+function tod_clins
+{
+ __clins_module_common || return $s_err
+}
diff --git a/module/test2/scheme.sh b/module/test2/scheme.sh
new file mode 100644
index 0000000..a83ee69
--- /dev/null
+++ b/module/test2/scheme.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+# holder.sh - test
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+declare -r test_t="\
+
+test2 - scheme.sh - test_t
+
+@@ START >>>>>>>>>>>>>
+${_TEST2_VAR0_HOLDER_}
+${_TEST2_VAR1_MODULE_}
+\${_TEST2_VAR2_EXTREF_}
+\${_TEST2_VAR3_EXTREF_}
+\${_TEST2_VAR4_EXTREF_}
+\${_TEST2_VAR5_EXTREF_}
+@@ END <<<<<<<<<<<<<<<
+"
diff --git a/tod b/tod
new file mode 100755
index 0000000..3d96645
--- /dev/null
+++ b/tod
@@ -0,0 +1,39 @@
+#!/bin/bash
+#
+# tod - main script
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+if (return 0 2>/dev/null); then
+ echo "Do not source, instead run ./tod" >&2
+ return 1
+fi
+
+set -e
+source util/util.sh
+source util/help.sh
+source util/validation.sh
+source util/io.sh
+source util/option.sh
+source config
+set +e
+
+option_process ${@}
diff --git a/util/help.sh b/util/help.sh
new file mode 100644
index 0000000..824584c
--- /dev/null
+++ b/util/help.sh
@@ -0,0 +1,122 @@
+#!/bin/bash
+#
+# help.sh
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+function __help
+{
+ cat <<EOF | less -RF
+`printf "\033[1m%s\033[0m\n" "NAME"`
+
+ tod - template open deploy
+
+`printf "\033[1m%s\033[0m\n" "USAGE"`
+
+ [ENVIRONMENT] tod [OPTION]=[MODULE,...]
+
+`printf "\033[1m%s\033[0m\n" "OPTIONS"`
+
+ --check Validate module: module.sh, holder.sh and scheme.sh
+ --watch Diff installed files against templates in scheme.sh
+ --fetch Download MANIFEST mirrors into \$mod_staging_d
+ --doall Make and install module files and \$instance_d files
+ --doins Make and install \$instance_d files, if applicable
+ ignore module files
+ --upall Update module files and \$instance_d files
+ --upins Update \$instance_d files, if applicable ignore module
+ files
+ --upmod Update module files only, do not touch \$instance_d files
+ --clins Uninstall \$instance_d files tracked in uninstall.sh
+ --clmod Remove \$mod_staging_d
+
+`printf "\033[1m%s\033[0m\n" "MODULES"`
+
+`for m in $(get_all_modules); do echo " $m"; done`
+
+`printf "\033[1m%s\033[0m\n" "ENVIRONMENT"`
+
+ LOG_D Default $LOG_D, 1 to show debug logs
+ LOG_W Default $LOG_W, 0 to suppress warning logs
+ LOG_I Default $LOG_I, 0 to suppress info logs
+ LOG_X Default $LOG_X, 0 to suppress logs from sub-process pipes
+ INSTANCE_DIR Default to \$script_dir, which defaults to the tod dir, set it
+ with the path to where an instance is to be created
+ HOST_NAME Default "$def_hostname", instance name
+ PASSWD_F Default null. Only needed for docker-compose and compose-cli.sh.
+ Path to a file containing runtime passwords. This file can be
+ removed after an instance is started
+ DEFANS Default null. Possible values are 'y' or 'n', allows to skip
+ user input when asked
+ MODALL Default 0, 1 to force the use of blacklisted modules. To
+ blacklist a module add it to BLMODULES in config. Blacklisted
+ modules are excluded by the list of modules run by default
+
+ blacklisted modules:
+
+`for m in ${BLMODULES[@]}; do echo " $m"; done`
+
+ INSTANCE_MODE Default "$debug", or "$release". Global switch to enable
+ debug or release version of an instance. Each module is
+ responsible for distinguishing between them. This option
+ can be overridden by module specific MODE. See example [4]
+
+ module specific MODE overrides INSTANCE_MODE:
+
+`for m in $(get_all_modules); do \\
+printf " %-14s %s\n" "${m^^}_MODE" \\
+"Default \\\$INSTANCE_MODE, \"$debug\" or \"$release\"";\\
+ done`
+
+`printf "\033[1m%s\033[0m\n" "EXAMPLES"`
+
+ [1] Call doall on every module, exclude blacklisted modules
+ ./tod --doall
+
+ [2] Force blacklist modules, call doall on test1 only
+ MODALL=1 ./tod --doall=test1
+
+ [3] Force blacklist modules, call doall on test1 and test2
+ MODALL=1 ./tod --doall=test1,test2
+
+ [4] Enable debug logs, make a release instance, but test2 debug
+ LOG_D=1 INSTANCE_MODE=release TEST2_MODE=debug ./tod --doall=test1,test2
+
+`printf "\033[1m%s\033[0m\n" "AUTHORS"`
+
+ Luigi Santivetti
+
+`printf "\033[1m%s\033[0m\n" "REPORTING BUGS"`
+
+ luigi.santivetti@gmail.com
+ https://bitbucket.org/luigi_s/tod
+
+`printf "\033[1m%s\033[0m\n" "VERSION"`
+
+ $credits_vers
+
+`printf "\033[1m%s\033[0m\n" "COPYRIGHT"`
+
+ Copyright 2020
+
+`printf "\033[1m%s\033[0m\n" "END"`
+EOF
+}
diff --git a/util/io.sh b/util/io.sh
new file mode 100644
index 0000000..5641c00
--- /dev/null
+++ b/util/io.sh
@@ -0,0 +1,507 @@
+#!/bin/bash
+#
+# io.sh
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# @1 : version file containg git sha
+# @return : stdout, instance version: '${git_sha}@${script_now}'
+function __get_script_ver_file
+{
+ [ "$1" = "-q" ] && local q=1 && shift
+
+ git diff-index --quiet HEAD 2>/dev/null || {
+ [ -n "$q" ] || lets -l -w "git unstaged changes"
+ }
+
+ [ -f "$instance_d/$__version" ] && {
+ local ver="$(< "$instance_d/$__version")"
+ local now="${ver##*@}"; local sha="${ver%%@*}"
+ [ ${#now} -eq ${#sha} ] && [ ${#now} -eq ${#script_now} ] || {
+ lets -l -e "invalid version"; return $s_err
+ }
+ echo "$ver" && return $s_ok
+ }
+ lets -l -w "invalid version file" && return $s_err
+}
+
+function __get_script_ver_git
+{
+ [ "$1" = "-q" ] && local q=1 && shift
+
+ git diff-index --quiet HEAD 2>/dev/null || {
+ [ -n "$q" ] || lets -l -w "git unstaged changes"
+ }
+
+ local -i len="${#script_now}"
+ local sha="$(git log --pretty=format:'%H' -n 1)"
+ [ -n "$sha" ] || { lets -l -e "git log failed"; return $s_err; }
+
+ echo "${sha:0:$len}@$script_now"
+}
+
+function get_script_ver
+{
+ case "$1" in
+ --file | -f ) shift; __get_script_ver_file "${@}" ;;
+ --git | -g ) shift; __get_script_ver_git "${@}" ;;
+ esac
+}
+
+# @1 : absolute path to prospective file
+# @return : file, instance version: '${git_sha}@${script_now}'
+function set_script_ver
+{
+ local -r vfile="$2"
+
+ is_unix_path $vfile && touch $vfile || {
+ lets -l -e "invalid version file"; return 1
+ }
+
+ echo "$1" > "$vfile"
+}
+
+# @1 : text to be sent to stdout via cat
+# @2 : (optional) absolute path to a prospective file in DOCKER_ROOT
+# @return : success if @2 isn't given or if @2 is valid
+# @desc : do not write to file if it doesn't live inside $instance_d
+function catvar
+{
+ if [ -z "$2" ]; then
+ cat <<EOF
+${1}
+EOF
+ elif is_in_tree_instance_d $2; then
+ echo > $2; cat > $2 <<EOF
+${1}
+EOF
+ else
+ return $s_err
+ fi
+}
+
+# @1 : text to append
+# @2 : absolute path to a valid file
+# @return : success if @2 is valid
+function appvar
+{
+ [ -f "$2" ] && catvar "$1" >> "$2" || lets -l -e "invalid: $2"
+}
+
+# @1 : (optional) absolute path to a valid file
+# @desc : echo a formatted header, including the file name if passed in
+function get_header
+{
+ if is_in_tree_instance_d "$1"; then
+ local header="\
+# $head_tag $(basename $1)
+
+$license"
+ else
+ local header="\
+$license"
+ fi
+ echo "$header"
+}
+
+# @desc : echo a formatted footer
+function get_footer
+{
+ local __ver="$(get_script_ver --file -q)"
+ __ver="${__ver%@*}"
+
+ local footer=\
+"# $foot_tag $__ver@$script_now"
+ echo "$footer"
+}
+
+function __create_instance_d
+{
+ is_valid_new_dir $2 || return $s_err4
+ mkdir -p $2 || return $s_err5
+
+ # do not remove directory - it should never be so
+ rm -f $1 || return $s_err6
+ ln -sf $2 $1 || return $s_err7
+}
+
+# @1 : absolute path to $instance_d
+# @desc : validate dkrc_out_d acutally is what we expect it to be
+function __is_valid_instance_d_link
+{
+ local ver_id="$(readlink -f $1)"
+ local version_f="$1/$__version"
+ ver_id="${ver_id##*-}"
+ local run_time="${ver_id#*@}"
+
+ test -n "$ver_id" || return $s_err3
+ is_unix_path "$1" || return $s_err4
+ test -L "$1" || return $s_err5
+ test -f "$version_f" || return $s_err6
+ [ "$ver_id" = "$(< $version_f)" ] || return $s_err7
+
+ # This is the case calling doins with multiple modules
+ [ "$run_time" = "$script_now" ] && return $s_err8 || \
+ return $s_ok
+}
+
+# @1 : absolute path to $instance_d
+# @desc : validate dkrc_out_d acutally is what we expect it to be
+function is_valid_instance_d_link
+{
+ __is_valid_instance_d_link $1
+ case "$?" in
+ $s_err3 ) lets -l -e "instance version id" ;;
+ $s_err4 ) lets -l -e "instance path" ;;
+ $s_err5 ) lets -l -e "instance link" ;;
+ $s_err6 ) lets -l -e "instance version file" ;;
+ $s_err7 ) lets -l -e "signature mismatch" ;;
+ $s_err8 ) return $s_err2 ;;
+ $s_ok ) return $s_ok ;;
+ esac; return $s_err
+}
+
+function __is_valid_input
+{
+ test -L "$1" && {
+ is_valid_instance_d_link "$1"
+ case "$?" in
+ $s_err2 ) return $s_err2 ;; # different module, same run
+ $s_err ) return $s_err ;; # genuine error
+ $s_ok )
+ local old_instance="$(basename "$(readlink -f $1)")"
+
+ lets -l -i "'$(basename $1)' points to '$old_instance'"
+ lets -l -i "update this link to '$2'"
+ lets -a "would you like to continue?" && \
+ return $s_ok || return $s_err ;;
+ esac
+ }
+
+ ! test -e "$1" || return $s_err3
+}
+
+# @desc : create a new instance_d, either by updating an existing
+# : symlink or creating everything from scratch.
+function create_instance_d
+{
+ local ver="$(get_script_ver --git)"
+ local new_instance="$(basename $1)-$ver"
+ local new_instance_d="$(dirname $1)/$new_instance"
+
+ __is_valid_input "$1" "$new_instance"
+ case "$?" in
+ $s_err ) lets -l -i "goodbye"; return $s_err ;;
+ $s_err2 ) return $s_ok ;; # no error, no further actions
+ $s_err3 ) lets -l -e "$1 busy"; return $s_err ;;
+ esac
+
+ __create_instance_d "$1" "$new_instance_d"
+ case "$?" in
+ $s_err4 ) lets -l -e "instance path" ; return $s_err ;;
+ $s_err5 ) lets -l -e "instance mkdir" ; return $s_err ;;
+ $s_err6 ) lets -l -e "instance unlink" ; return $s_err ;;
+ $s_err7 ) lets -l -e "instance link" ; return $s_err ;;
+ esac
+
+ set_script_ver $ver $instance_d/$__version || return $s_err
+}
+
+# @ : a list of absolute paths to prospective folders
+function create_directory
+{
+ local d
+
+ for d in ${@}; do
+ is_unix_path $d || { lets -l -d "invalid path: $d"; return $s_err; }
+ [ -d "$d" ] || { mkdir -p $d && lets -l -d "mkdir $d"; }
+ done
+}
+
+# @1 : (optional) option --dry
+# @2 : file text content
+# @3 : absolute path to a valid target file
+# @return : Success if @3 is valid
+# @desc : Redirect a bash variable to stdout or file
+function __write_to_file
+{
+ [ "$1" = "--dry" ] && shift || local file_path=$2
+ local file_text="${!1}"
+
+ case "$1" in
+ *"_bang_"* )
+ local shebang="$(catvar "$file_text" | head --lines 1)"
+ file_text="$(catvar "$file_text" | tail --lines +2)"
+ local file_content="\
+${shebang}
+
+$(get_header $2)
+
+${file_text}
+
+$(get_footer)"
+ ;;
+ * )
+ local file_content="\
+$(get_header $2)
+
+${file_text}
+
+$(get_footer)"
+ ;;
+ esac
+
+ catvar "$file_content" $file_path
+}
+
+# @1 : diff flags
+# @2 : file text content
+# @3 : absolute path to an existing file
+# @desc : This function can diff an existing file against some new content
+function __diff_file
+{
+ local flags=$1; shift
+
+ if [ -f "$2" ]; then
+ # NOTE: "$(echo "$1")" makes sure $1 isn't broken down into multiple
+ # strings according to the bash IFS expasion.
+ # FIXME: check __write_to_file
+ diff $flags $2 <(__write_to_file --dry "$(echo "$1")" $2)
+ else
+ lets -l -d "invalid file: $2"
+ return $s_err2
+ fi
+}
+
+# @1 : diff flags
+# @2 : file text content
+# @3 : absolute path to an existing file
+# @desc : This function can diff an existing file against some new content
+function is_file_diff
+{
+ __diff_file "$1" "$2" "$3" &>/dev/null
+}
+
+# @1 : file text content
+# @2 : absolute path to a valid file
+function __splash_file
+{
+ local flags="--suppress-common-lines"
+ flags+=" --ignore-matching-lines=$rex_legal_file_header_tag"
+ flags+=" --ignore-matching-lines=$rex_legal_file_footer_tag"
+ flags+=" --report-identical-files"
+ flags+=" --color"
+# flags+=" --ignore-blank-lines"
+
+ __diff_file "$flags" "$1" "$2" | lets -l -x "diff"
+ case "${PIPESTATUS[0]}" in
+ $s_ok ) lets -l -i --phew "MATCH: $(basename "$2")" ;;
+ $s_err ) lets -l -w "DIFFER: $(basename "$2")" ;;
+ $s_err2 ) lets -l -w "NOT-FOUND: $(basename "$2")" ;;
+ esac
+
+ return $s_ok
+}
+
+# @1 : file text content
+# @2 : absolute path to a valid file
+# @desc : Update the content of an already generated file. This is useful when
+# : the user doesn't want/need to recreate a new rootfs from scratch.
+function __update_file
+{
+ local file_name="$(basename $2)"
+ local flags="--suppress-common-lines"
+ flags+=" --ignore-matching-lines=$rex_legal_file_header_tag"
+ flags+=" --ignore-matching-lines=$rex_legal_file_footer_tag"
+ #flags+=" --report-identical-files"
+ #flags+=" --ignore-blank-lines"
+
+ is_file_diff "$flags" "$1" $2
+ case "$?" in
+ $s_ok ) lets -l -i "no need to update: $file_name"; return $s_ok ;;
+ $s_err2 ) lets -l -w "add one on update: $file_name" ;;
+ esac
+
+ local old_foot_tags="$(grep -E "$rex_legal_file_footer_tag" -- $2 2>/dev/null)"
+
+ if ! __write_to_file "$1" $2; then
+ lets -l -e "failed to update $file_name"; return $s_err
+ fi
+
+ lets -l -i "changed on update: $file_name"
+
+ if [ -n "$old_foot_tags" ]; then
+ if ! appvar "$old_foot_tags" $2; then
+ lets -l -e "failed to append $old_foot_tags"
+ fi
+ fi
+}
+
+function __create_file
+{
+ [ "$1" = "--dry" ] && local dry=$1 && shift || local dry=""
+
+ if [ -f "$2" ]; then
+ lets -l -w "overriding: $(basename $2)"
+ fi
+
+ __write_to_file $dry "$1" "$2"
+}
+
+function __file_do_helper
+{
+ local __file="$2"
+ local __cmd="rm"
+ local __line="$__cmd $__file $script_now $module $1"
+
+ case "$1" in
+ --update )
+ [ -f "$3" ] && {
+ local flags="--follow-symlinks -i -E"
+
+ # Has $2 been actually updated?
+ grep -qE "$rex_legal_file_footer_tag$script_now" $2 && {
+
+ # $3 is not marked for deletion
+ grep -qE "^#$__cmd $__file .*" $3 && {
+
+ # Update line to report script_now, module and option
+ sed $flags "s%^#$__cmd $__file .*$%#$__line%g" -- $3
+ return $s_ok
+ }
+
+ # $3 is marked for deletion
+ grep -qE "^$__cmd $__file .*" $3 && {
+
+ # Update line to report script_now, module and option.
+ # Also, comment it out, as $3 is not to be removed.
+ sed $flags "s%^$__cmd $__file .*$%#$__line%g" -- $3
+ return $s_ok
+ }
+
+ [ "$(grep -E "$rex_legal_file_footer_tag" $2 | wc -l)" -eq 1 ] && {
+
+ # Update instance by adding a file. Equivalent to --create, but
+ # do not mark $2 for removal.
+ [ -f "$3" ] && {
+ ! grep -q "$__cmd $2" $3 && echo "#$__line" >> $3
+ return $s_ok
+ }
+
+ lets -l -w "update without $3. Advise to wipe it all"
+ echo "#$__line" > $3 && return $s_ok
+ }
+
+ lets -l -e "$(basename $3) not tracking $(basename $2)"
+ return $s_err
+ }
+
+ # Is $2 marked for deletion? If so, un-mark it
+ grep -qE "^$__cmd $__file .*" $3 && {
+ sed $flags "s%^$__cmd $__file%#$__cmd $__file%g" -- $3
+ return $s_ok
+ }
+ } ;;
+ --create )
+ [ -f "$3" ] && {
+ ! grep -q "$__cmd $2" $3 && echo "$__line" >> $3
+ return $s_ok
+ }; echo "$__line" > $3 ;;
+ esac
+}
+
+# @1 : file text content
+# @2 : absolute path to a valid file
+# @desc : Update the content of an already generated file. This is useful
+# : when the user doesn't want/need to recreate a new rootfs from
+# : scratch.
+function __file_do
+{
+ # Option
+ local option=$1; shift
+ local __un_f=$1; shift
+
+ case "$option" in
+ --update ) local -r func="__update_file" ;;
+ --create ) local -r func="__create_file" ;;
+ --splash ) local -r func="__splash_file" ;;
+ *) lets -l -e "invalid option $option"; return $s_err ;;
+ esac
+
+ local -i len="${#} / 2"
+ local -i i
+
+ local -a path=( "${@:1:$len}" ); shift $len
+ local -a cont=( "${@}" )
+
+ if [ ${#path[@]} -ne ${#cont[@]} ]; then
+ lets -l -e "mismatch: $len, ${#path[@]} vs ${#cont[@]}"
+ return $s_err
+ fi
+
+ # Remove comment before processing all files, for this module only
+ if [ -f "$__un_f" ]; then
+ cp "$__un_f" "$__un_f.bkp" # be safe
+ sed --follow-symlinks -i -E "s%^#(.+ $module .+)%\1%g" -- $__un_f
+ fi
+
+ for i in ${!path[@]}; do
+ if ! $func ${cont[$i]} ${path[$i]}; then
+ lets -l -d "failed to ${func%_file} ${path[$i]}"
+ cp "$__un_f.bkp" "$__un_f" && rm -f "$__un_f.bkp"
+ return $s_err
+ fi
+
+ # FIXME: handle file permissions in a proper way
+ case "${cont[$i]}" in *"_bang_"* ) chmod +x ${path[$i]} ;; esac
+
+ __file_do_helper $option ${path[$i]} $__un_f || return $s_err
+ lets -l -d "${func%_file} $(basename ${path[$i]})"
+ done
+
+ case "$option" in
+ --update )
+ [ ! -f "$__un_f" ] && {
+ cp "$__un_f.bkp" "$__un_f" && rm -f "$__un_f.bkp"; return $s_err
+ }
+
+ local -a __rmline
+ while read -a __rmline; do
+ local __cmd="${__rmline[0]}"; local __f="${__rmline[1]}"
+ case "${__rmline[*]}" in
+ '' | '\n' | '#'* )
+ lets -l -d "skip: $__cmd: $(basename "$__f")"
+ continue ;;
+ * ) [ ! -f "$__f" ] && {
+ lets -l -w "$__cmd *not* installed: $(basename "$__f")"
+ } || { lets -l -i "$__cmd installed: $(basename "$__f")"; }
+
+ eval "$__cmd $__f" && {
+ # Got to remove this line and it will be lost forever
+ sed --follow-symlinks -i -E "\:$__cmd $__f.*:d" -- "$__un_f"
+ } || { lets -l -e "failed to remove $__f"; } ;;
+ esac
+ done <<< $(grep -E "[[:space:]]$module[[:space:]]" -- $__un_f) ;;
+ esac
+
+ rm -f "$__un_f.bkp"
+ return $s_ok
+}
diff --git a/util/option.sh b/util/option.sh
new file mode 100644
index 0000000..5f13a4c
--- /dev/null
+++ b/util/option.sh
@@ -0,0 +1,218 @@
+#!/bin/bash
+#
+# shared.sh - multiple utils
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+function __call_module
+{
+ local -r this_module="$1"
+
+ [ -n "$this_module" ] || { lets -l -e "module not defined"; return $s_err; }
+
+ if is_opt "upins" "upall"; then
+ [ ! -d "$instance_d" ] && {
+ lets -l -e "trying to update an invalid instance"
+ return $s_err
+ }
+ fi
+
+ if is_opt "check" "watch" "upins" "upall" "doins" "doall"; then
+ local __envtable; local -i ret
+
+ __envtable="$(get_module_dependencies "envtable" $this_module)"; ret=$?
+ case "$ret" in
+ $s_ok )
+ [ -n "$__envtable" ] && {
+
+ # Export what's inside __envtable and check the actual table is
+ # imported as a variable.
+ eval "$__envtable" && declare -p -- "envtable" >/dev/null || {
+ lets -l -e "failed to export enviroment table"
+ return $s_err
+ }
+
+ # envtable is successfully on the stack, throw away its
+ # textual representation.
+ unset __envtable
+ } || {
+ lets -l -e "failed to get environment table"
+ return $s_err
+ } ;;
+ * ) return $s_err ;;
+ esac
+ fi
+
+ if is_opt "check"; then
+ (
+ # Export external variables found
+ set -eu; eval "${envtable[$this_module]}"; set +eu
+
+ case "${#deps[@]}" in
+ 0 ) lets -l -i "$this_module doesn't have dependencies"
+ return $s_ok ;;
+ * ) lets -l -i "$this_module depends on: ${deps[@]}"
+ are_in_modules ${deps[@]} || {
+ lets -l -w "${deps[@]} not in MODULES"
+ } ;;
+ esac
+
+ # Check if this module correctly defines its own
+ # depmod. Return with an error if it doesn't since
+ # other options rely on this module depmod.
+
+ source "$module_d/$this_module/$module_sh"
+ if ! declare -p -- "depmod" &>/dev/null; then
+ lets -l -e "$this_module must define 'depmod'"
+ return $s_err
+ fi
+
+ for dep in ${deps[@]}; do
+ __is_in_array $dep ${depmod[@]} || {
+ lets -l -e "$dep not in $this_module depmod: ${depmod[@]}"
+ lets -l -e "add $dep to $this_module depmod"
+ return $s_err
+ }
+ done
+ )
+ return $?
+ fi
+
+ (
+ local -a files=( )
+ local callback
+
+ files+=( "$module_d/$this_module/$module_sh" )
+ files+=( "$module_d/$this_module/$holder_sh" )
+ files+=( "$module_d/$this_module/$scheme_sh" )
+
+ if is_opt "doins" "upins" "upall" "doall" "watch"; then
+
+ # Export external references
+ if test_source <(echo "${envtable[$this_module]}"); then
+ eval "${envtable[$this_module]}" || return $s_err2
+ else
+ return $s_err2
+ fi
+ fi
+
+ # Source needed files
+ if test_source "${files[@]}"; then
+ for f in ${files[@]}; do source "$f" || return $s_err3; done
+ else
+ return $s_err3
+ fi
+
+ callback="$(declare -F -- "tod_$2" 2>/dev/null)" || return $s_err6
+
+ if is_opt "doins" "doall"; then
+ create_instance_d $instance_d || return $s_err4
+ fi
+
+ # -- Here be dragons
+ eval "$callback \"${@:2}\"" || return $s_err5
+ )
+ case "$?" in
+ $s_err2 ) lets -l -e "$this_module: failed to export environment" ;;
+ $s_err3 ) lets -l -e "$this_module: failed to source $f" ;;
+ $s_err4 ) lets -l -e "$this_module: failed to create instance_d" ;;
+ $s_err5 ) lets -l -e "$this_module: $callback failed" ;;
+ $s_err6 ) lets -l -w "$this_module: $callback not available"
+ return $s_ok ;;
+ $s_ok ) return $s_ok ;;
+ esac
+
+ return $s_err
+}
+
+function __option_call
+{
+ local option="$1"; shift
+ local -a mods=( "${@}" )
+ local modname
+
+ is_valid_module "${mods[@]}"
+ case "$?" in
+ $s_err | $s_off_broken ) return $s_err ;;
+ $s_ok | $s_off_usable ) ;;
+ esac
+
+ # Actually call into module code
+ for modname in ${OPTARG[@]}; do
+ __call_module $modname $option
+ [ $? -eq $s_ok ] || return $s_err
+ done
+}
+
+function option_call
+{
+ local -a mods
+
+ mods=( $(get_modules_optarg "${@:2}") ) || return $s_err
+ # Filtered out list of requested modules
+ OPTARG=( "${mods[@]}" )
+
+ __option_call "$1" "${mods[@]}" >/dev/null
+}
+
+# @desc : process command line options only two options allowed per invocation
+# : at the moment. Keep it simple, keep it robust. For options details
+# : see help.sh.
+# :
+# : credits to https://stackoverflow.com/a/28466267
+function __option_process
+{
+ local opt; OPTIND=1
+
+ # Be quiet and disallow any single char option
+ local -r options=":-:"
+ local -a args
+
+ if getopts "$options" opt; then
+ if [ "$opt" = "-" ]; then
+ opt="${OPTARG%%=*}" # Get option name
+ OPTARG="${OPTARG#$opt}" # Get option argument(s)
+ OPTARG="${OPTARG#=}" # Drop `=` if any
+ OPTARG="${OPTARG//,/ }" # Drop `,` if any
+
+ # Early return for help
+ [ "$opt" = "help" ] && __help && return $s_ok
+
+ is_in_options "$opt" || return $s_err2
+ fi
+ fi
+
+ OPTIONS[$opt]=1; option_call $opt "${OPTARG[@]}"
+ case "$?" in
+ $s_err ) lets -l -e "$opt failed"; return $s_err ;;
+ $s_ok ) lets -l -i --phew "$opt success"; return $s_ok ;;
+ esac
+}
+
+function option_process
+{
+ __option_process ${@}
+ case "$?" in
+ $s_err2 ) lets -l -e "invalid option, --help" ;;
+ $s_err ) return $s_err ;;
+ $s_ok ) return $s_ok ;;
+ esac
+}
diff --git a/util/util.sh b/util/util.sh
new file mode 100755
index 0000000..6bd5289
--- /dev/null
+++ b/util/util.sh
@@ -0,0 +1,699 @@
+#!/bin/bash
+#
+# util.sh
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# Set log line max width
+if [ -n "$COLUMNS" ]; then
+ declare -ir prn_wmax="$COLUMNS"
+elif [ -n "$(tput cols 2>/dev/null)" ]; then
+ declare -ir prn_wmax="$(tput cols)"
+else
+ declare -ir prn_wmax="135" # my size :)
+fi
+
+# Set width ration for description
+if [ "${LOG_D:-0}" -eq "1" ]; then
+ declare -ir prn_wdesc_ratio="35"
+ # Set log description max width
+ declare -ir prn_wdesc="$prn_wmax / 100 * $prn_wdesc_ratio"
+else
+ declare -ir prn_wdesc="0"
+fi
+
+# Set log tags
+declare -Ar prn_tag=( \
+ ["W"]="WARNING" ["E"]="ERROR" ["D"]="DEBUG" \
+ ["I"]="INFO" ["A"]="ASKING" ["X"]="PIPE" \
+)
+
+# Set log tag max width
+declare -ir prn_wtag="$(for t in ${prn_tag[@]}; do \
+ echo -ne $t | wc -m; done | sort -r | head -1)"
+
+# Set log offset for symbols and whitespaces
+declare -ir prn_wofs="8"
+
+# Set log message max width
+declare -ir prn_wmsg="$prn_wmax - $prn_wdesc - $prn_wtag - $prn_wofs"
+
+# Some color
+declare -rA c=( \
+ [LGREEN]="\033[1;32m" [LGREY]="\033[0;37m" [LCYAN]="\033[96m" \
+ [YELLOW]="\033[1;33m" [LRED]="\033[1;31m" [LBLUE]="\033[1;34m" \
+ [NONE]="\033[0m" \
+)
+
+if which fold &>/dev/null; then
+ function prn_line
+ {
+ echo -en "$1\n" | fold -s -w "$prn_wmsg"
+ }
+else
+ function prn_line
+ {
+ echo -e "$1\n"
+ }
+fi
+
+# @return : stdout, absolute path to the main script, assume it is one level up '..'
+function get_script_dir
+{
+ pushd \
+ "$(dirname "$(readlink -f ${BASH_SOURCE[0]} 2>/dev/null)" 2>/dev/null)" \
+ &>/dev/null && {
+ echo "$(realpath "$(pwd)"/..)"
+ popd &>/dev/null
+ }
+}
+
+# @return : stdout, runtime timestamp at lauching time
+function get_script_now
+{
+ local now="$(date +'%d%m%Y%H%M%S')" && echo "$now"
+}
+
+# @desc : main routine for handling basic stdout messages
+function prn
+{
+ local nl colr ltag lmsg desc line fdesc ftag fmt_ltag fmt
+
+ fdesc="^"
+ ftag="^"
+
+ # Order here should match the invocation order in log()
+ while [ ${#} -gt 0 ]; do
+ case "$1" in
+ -n) nl="\n"; shift ;;
+ -d) desc="$2"; shift 2 ;;
+ -t) ltag="$2"; shift 2 ;;
+ -c) colr="$2"; shift 2 ;;
+ -x) ftag="$2"; line="$2\n"; shift 2 ;;
+ -l) line="$(prn_line "$line$2\n")"; shift 2 ;;
+ *) shift ;;
+ esac
+ done
+
+ # Build the format output string
+ fmt_ltag="$colr%*s${c[NONE]}"
+ fmt="%*.*s $fmt_ltag %.*s$nl"
+
+ # Process stdout
+ while read -r lmsg; do
+ printf "$fmt" $prn_wdesc $prn_wdesc $desc \
+ $prn_wtag ${ltag:0:$prn_wtag} $prn_wmsg "$lmsg"
+
+ # If printing folded lines, remove func and tag
+ desc=$fdesc && ltag=$ftag
+ done < <(cat <<EOF
+$line
+EOF
+ )
+}
+
+# @1 : text to be sent to stdout via prn
+# @return : Success if user enters y, error if n, otherwise it keeps
+# : asking for input.
+function ask
+{
+ local desc="${FUNCNAME[2]}:${BASH_LINENO[1]}"
+ local ans=""
+
+ [ "$answer" = 'y' ] && return $s_ok
+ [ "$answer" = 'n' ] && return $s_err
+
+ prn -d $desc -t ${prn_tag[A]} -c ${c[LBLUE]} -l "${*} [y/n]" 1>&2
+ while [ "$ans" != 'y' ] && [ "$ans" != 'n' ]; do read -s -n 1 ans; done
+
+ printf " $ans\n" 1>&2; [ "$ans" = "y" ] && return $s_ok || return $s_err
+}
+
+# @ : list of options and arguments (-x, -w, -d, -e, -i and --phew). For
+# : -d, debug and -i, info, --phew overrides the color to green.
+# @desc : main routine for logging facilities. Note, all logs are redirected
+# : to stderr and logging error (-e) returns an error (return 1)
+function log
+{
+ local -i do_print=0
+ local -a msg=( )
+ local desc=""
+ local ltag=""
+ local colr=""
+ local xlog=""
+
+ [ $LOG_D -eq 1 ] && desc="-d ${FUNCNAME[2]}:${BASH_LINENO[1]}" || desc="-d ''"
+
+ case "${1}" in
+ -x) [ $LOG_X -eq 1 ] && {
+ shift
+ [ -n "$1" ] && xlog="-x $1"
+ shift
+ colr="-c ${c[LCYAN]}"
+ do_print=1
+ msg=( "${*:-$(</dev/stdin)}" )
+ ltag=${prn_tag[X]}
+ } ;;
+ -w) [ $LOG_W -eq 1 ] && {
+ shift
+ colr="-c ${c[YELLOW]}"
+ do_print=1
+ msg=( "${*}" )
+ ltag=${prn_tag[W]}
+ } ;;
+ -d) [ $LOG_D -eq 1 ] && {
+ shift
+ [ "$1" = "--phew" ] && {
+ shift
+ colr="-c ${c[LGREEN]}"
+ } || colr="-c ${c[LGREY]}"
+ do_print=1
+ msg=( "${*}" )
+ ltag=${prn_tag[D]}
+ } ;;
+ -i) [ $LOG_I -eq 1 ] && {
+ shift
+ [ "$1" = "--phew" ] && {
+ shift
+ colr="-c ${c[LGREEN]}"
+ } || colr="-c ${c[NONE]}"
+ do_print=1
+ msg=( "${*}" )
+ ltag=${prn_tag[I]}
+ } ;;
+ -e) # Always print errors
+ shift
+ prn -n $desc -t ${prn_tag[E]} -c ${c[LRED]} -l "${*}" 1>&2
+ return $s_err ;;
+ esac
+
+ [ $do_print -eq 1 ] && {
+ prn -n $desc -t $ltag $colr $xlog -l "${msg[*]}" 1>&2
+ }; return $s_ok
+}
+
+function is_set
+{
+ [ "$(set -o | grep $1 | grep -oE "(on|off)")" = "on" ]
+}
+
+# desc : util handler
+function lets
+{
+ is_set "xtrace" && local xtrace="set -x" && set +x
+ local -i ret
+
+ case "$1" in
+ -l|--log) shift && log ${@}; ret=$? ;;
+ -a|--ask) shift && ask ${@}; ret=$? ;;
+ -d|--die) shift && die ${@}; ret=$? ;;
+ *) eval ${xtrace}; return $s_err ;;
+ esac
+
+ eval ${xtrace}; return $ret
+}
+
+# @1 : string compliant with unix file system node addressing
+# @desc : using ${*} to get it to fail if passed in with multiple words
+function is_unix_path
+{
+ [ ! -z "${*}" ] && pathchk -P -- "${*}" &>/dev/null
+}
+
+# @1 : absolute path to file
+# @2 : absolute path to directory
+# @return : success if @2 exists and @1 is a valid unix path and @1 is a node in @2,
+# : error otherwise.
+# @desc : useful when @1 is a prospective file, but not a file just yet
+function is_in_tree
+{
+ [ -d "${2}" ] && is_unix_path "${1}" || return 1
+
+ case "${1}" in "${2}"*) return 0 ;; *) return 1 ;; esac
+}
+
+# @1 : absolute path to a prospective file
+# @return : success if the named file in @1 is or will be in the $instance_d tree
+function is_in_tree_instance_d
+{
+ is_in_tree "${1}" "${instance_d}"
+}
+
+# @1 : file mode permission
+# @2 : file owner and group mode permission
+# @3 : (optional) recursive flag -R
+# @ : list of files and/or directories
+# @return : Success if it could chmod and chown a valid file
+function set_mode
+{
+ local mode=$1; shift
+ local own=$1; shift
+ local t
+
+ if [ "$1" = "-R" ]; then local flags=$1; shift; fi
+
+ for t in ${@}; do
+ if [ -f "$t" ] || [ -d "$t" ]; then
+ sudo chmod $flags $mode "$t" && sudo chown $own:$own "$t" || return 1
+ else
+ lets -l -e "invalid: $t"
+ fi
+ done
+}
+
+# @1 : list of packages
+function util_install_dependency
+{
+ local pkg
+
+ for pkg in ${@}; do
+ sudo apt-get install -y --fix-missing $pkg 2>&1 | lets -l -x "apt-get"
+ [ ${PIPESTATUS[0]} -eq 0 ] || {
+ lets -l -e "failed to install $pkg"; return 1
+ }
+ done
+}
+
+# @1 : absolute path
+# @desc : Validate a prospective directory path
+function is_valid_new_dir
+{
+ is_unix_path "${*}" && [ ! -e "${*}" ]
+}
+
+# @desc : remove trailing and leading whitespaces
+function get_wtrim
+{
+ echo "${*}"
+}
+
+# @1 : absolute path to a generic file
+# @return : basename without extension
+# @desc : convert an absolute path to a lhs suitable variable name. This
+# : means stripping anything after the dot
+function get_stripname
+{
+ local stripped
+
+ stripped="$(basename $1)"; stripped="${stripped%%.*}"
+ echo "$stripped"
+}
+
+# @1 : array variable name
+# @desc : check if the vriable named in @1 is set and is an array
+function is_array_set
+{
+ declare -p -- "${*}" | grep -q "declare \-a"
+}
+
+# @1 : function name
+# @desc : check if the vriable named in @1 is set
+function is_function_set
+{
+ declare -F "${*}" &>/dev/null
+}
+
+function is_system_ready
+{
+ case "$BASH_VERSION" in
+ ''|[123].*|4.[0123]) echo "ERROR: Bash 4.4 required" >&2; return 1 ;;
+ esac
+}
+
+# @1 : asoblute path to output file
+# @2 : fd variable name reference
+# @3 : char representing file mode operation - r,w,rw
+# @desc : in preparation to add support for dedicated fds for logs and output
+function fdopen
+{
+ local -n out_fd=$2
+ local -r file=$1
+
+ [ -z "$out_fd" ] || { lets -l -e "invalid out fd"; return 1; }
+
+ case "$3" in
+ rw|wr) [ -n "$file" ] && [ ! -f "$file" ] && \
+ is_unix_path $file && exec {fd}<> $file ;;
+ w ) [ ! -f "$file" ] && \
+ is_unix_path $file && exec {fd}> $file ;;
+ r ) [ -r "$file" ] && exec {fd}< $file ;;
+ esac
+
+ # Posix bash dynamic fd allocation is from 9 on
+ [ -f "$file" ] && [ $fd -gt 9 ] && out_fd=$fd && return
+
+ lets -l -e "failed to open fd=$1"
+ return 1
+}
+
+# @1 : fd int
+# @2 : char representing file mode operation - r,w,rw
+# @desc : in preparation to add support for dedicated fds for logs and output
+function fdclose
+{
+ local -ri fd=$1
+
+ [ $fd -gt 9 ] || { lets -l -e "invalid file fd"; return 1; }
+
+ case "$2" in
+ rw|wr) exec {fd}>&- && { lets -l -d "$fd closed"; return; }
+ exec {fd}<&- && { lets -l -d "$fd closed"; return; } ;;
+ w ) exec {fd}>&- && { lets -l -d "$fd closed"; return; } ;;
+ r ) exec {fd}<&- && { lets -l -d "$fd closed"; return; } ;;
+ esac
+
+ lets -l -e "failed to close fd=$2"
+ return 1
+}
+
+function get_hunk_indexes
+{
+ local begin="$rex_legal_manifest_hook_begin"
+ local ended="$rex_legal_manifest_hook_ended"
+ local -a idx
+
+ idx=( $(grep -n -e "$begin" -e "$ended" $manifest_f | cut -f1 -d':') )
+ local -i mod="${#idx[@]} % 2"
+
+ [ $mod -eq 0 ] && echo ${idx[@]} || lets -l -e "invalid $manifest_f"
+}
+
+function get_hunk_pair
+{
+ local -i head; local -i tail; local -a pair
+ local -i from=$1; shift
+
+ pair=( ${@:$from:2} )
+ [ ${#pair[@]} -eq 2 ] || { [ ${#pair[@]} -eq 0 ] && return $s_ok; } || \
+ return $s_err
+
+ tail=${pair[0]}; head=${pair[1]};
+ tail="$head - $tail + 1"
+
+ echo "$head $tail"
+}
+
+function get_hunk
+{
+ cat $manifest_f | head -n $1 | tail -n $2
+}
+
+# @1 : remote address
+# @2 : branch name
+# @3 : mirror name
+function get_new_sha_upstream
+{
+ local remote_sha
+
+ remote_sha="$(git ls-remote $1 $2 | head -1 | cut -f 1)"
+
+ if [ -d "$3" ]; then
+ [ -f "$3/$__version" ] || {
+ lets -l -e "${3:-uknown mirror} not versioned"
+ return $s_err2
+ }
+ else
+ mkdir -p "$3" || { lets -l -e "mkdir failed: $3"; return $s_err2; }
+ echo $remote_sha; return $s_ok
+ fi
+
+ [ "$(< "$3/$__version")" != "$remote_sha" ] && {
+ lets -l -d "$3: $remote_sha"
+ rm -rf $3; echo $remote_sha; return $s_ok
+ } || {
+ lets -l -i "$3 is up to date";
+ }
+
+ return $s_err
+}
+
+function __git_archive
+{
+ local mirror="$1"
+ local remote="$4"
+ local staging_d="$(dirname $2)"
+ local flags="--prefix=$mirror/" # need trailing slash
+ flags+=" --output=$2"
+ flags+=" --format=$3"
+ flags+=" --remote=$remote"
+# flags+=" --verbose"
+ local branch="$5"
+ local remote_sha
+
+ remote_sha="$(get_new_sha_upstream $remote $branch $staging_d/$mirror)"
+ case "$?" in
+ $s_err2 ) return $s_err ;; # no new sha, an error occurred
+ $s_err ) return $s_ok ;; # no new sha, no need to update
+ esac
+
+ git archive $flags $branch 2>&1 | lets -l -x "git"
+
+ [ ${PIPESTATUS[0]} -eq $s_ok ] && {
+ tar -xf $2 -C "${2%/*}" 2>&1 | lets -l -x "tar"
+ [ ${PIPESTATUS[0]} -eq $s_ok ] && rm -rf $2 || {
+ lets -l -w "tar failed: $2. Clean up needed"
+ return $s_err
+ }
+ echo $remote_sha > "${2%/*}/$mirror/$__version"
+ }
+
+ return ${PIPESTATUS[0]}
+}
+
+function __git_clone
+{
+ local mirror_d="$3"
+ local branch="$1"
+ local remote="$2"
+ local flags="--single-branch"
+ flags+=" --depth=1"
+ flags+=" --branch=$branch"
+ flags+=" --verbose"
+ local remote_sha
+
+ remote_sha="$(get_new_sha_upstream $remote $branch $mirror_d)"
+ case "$?" in
+ $s_err2 ) return $s_err ;; # no new sha, an error occurred
+ $s_err ) return $s_ok ;; # no new sha, no need to update
+ esac
+
+ git clone $flags $remote $3 2>&1 | lets -l -x "git"
+ [ ${PIPESTATUS[0]} -eq $s_ok ] || return $s_err
+
+ echo $remote_sha > "$mirror_d/$__version"
+}
+
+function __wget_file
+{
+ local output="-O $1"
+ local remote="$2"
+ local flags="-nv"
+
+ if [ ! -f "$1" ]; then
+ wget $flags $output $remote 2>&1 | lets -l -x "wget"
+ [ ${PIPESTATUS[0]} -eq 0 ] || {
+ lets -l -d "failed wget $(basename $1)"
+ return $s_err
+ }
+ else
+ lets -l -i "$1 up to date"
+ fi; [ -f "$1" ]
+}
+
+# @1 : module name
+# @2 : output dir
+function process_manifest
+{
+ local -i from=1 # $@ is 1 indexed
+ local -a hunks
+ local -a pair
+
+ [ -z "$1" ] && { lets -l -e "invalid module name"; return $s_err; }
+
+ hunks=( $(get_hunk_indexes) )
+ case "$?" in $s_err ) return $s_err ;; esac
+
+ pair=( $(get_hunk_pair $from ${hunks[@]}) )
+ case "$?" in $s_err ) return $s_err ;; esac
+
+ while [ -n "$pair" ]; do
+ (
+ # Clean up manifest fields before sourcing
+ modname=""; mirror=""; protocol=""; branch=""; host=""; user="";
+ method=""; format=""; file=""
+
+ set -e
+ source <(get_hunk ${pair[@]}) && set +e || { set +e; return $s_err3; }
+
+ [ "$1" = "$modname" ] || return $s_ok # no error, try again
+
+ case "$method" in
+ git-archive )
+ [ -n "$user" ] && user="$user@"
+ [ -n "$protocol" ] && protocol="$protocol://"
+ [ -n "$format" ] || return $s_err4
+
+ local remote="${protocol:-}${user:-}$host/$mirror"
+ local tarball="$2/$mirror.$format"
+
+ __git_archive \
+ $mirror $tarball $format $remote $branch || return $s_err5 ;;
+ git-clone )
+ [ -n "$user" ] && user="$user@"
+ [ -n "$protocol" ] && protocol="$protocol://"
+
+ local remote="${protocol:-}${user:-}$host/$mirror"
+
+ __git_clone $branch $remote $2/$mirror || return $s_err6 ;;
+ wget-file )
+ [ -n "$file" ] || return $s_err7
+ [ -n "$format" ] && file="$file.$format"
+
+ __wget_file $2/$file $host/$file || return $s_err8 ;;
+ *) return $s_err9 ;;
+ esac
+ ) || {
+ case "$?" in
+ $s_err3 ) lets -l -e "$1: failed to source manifest" ;;
+ $s_err4 ) lets -l -e "$1: manifest missing format" ;;
+ $s_err5 ) lets -l -e "$1: failed git archive" ;;
+ $s_err6 ) lets -l -e "$1: failed git clone" ;;
+ $s_err7 ) lets -l -e "$1: manifest missing file" ;;
+ $s_err8 ) lets -l -e "$1: failed wget file" ;;
+ $s_err9 ) lets -l -e "$1: invalid manifest method" ;;
+ esac; return $s_err
+ }
+
+ from="$from + 2"; pair=( $(get_hunk_pair $from ${hunks[@]}) )
+ [ $? -eq 0 ] || return $s_err
+ done; return $s_ok
+}
+
+function get_all_modules
+{
+ ls -Ad $module_d/*/ | xargs -n1 basename
+}
+
+function get_modules_optarg
+{
+ local modname; local -a mods=( ); local -a imods=( )
+
+ if [ -z "${*}" ]; then
+ imods=( $(get_all_modules) )
+ else
+ imods=( "${@}" )
+ fi
+
+ for modname in ${imods[@]}; do
+ [ -n "$modname" ] && [ -d "$module_d/$modname" ] || {
+ lets -l -w "$modname not found"
+ continue
+ }
+
+ __is_in_array "$modname" "${BLMODULES[@]}" && [ $modall -eq 0 ] && {
+ lets -l -d "$modname is blacklisted, set MODALL=1 to force"
+ continue
+ }
+
+ mods+=( $modname )
+ done
+
+ [ -n "${mods[*]}" ] && echo "${mods[@]}"
+}
+
+function is_opt
+{
+ local k
+
+ for k in ${!OPTIONS[@]}; do
+ [ ${OPTIONS[$k]} -eq 1 ] && \
+ case "${*}" in *"$k"* ) return $s_ok ;; esac
+ done
+
+ return $s_err
+}
+
+function get_opt
+{
+ local k
+
+ for k in ${!OPTIONS[@]}; do
+ [ ${OPTIONS[$k]} -eq 1 ] && echo $k
+ done
+}
+
+function __is_in_array
+{
+ printf "%s\n" "${@:2}" | grep -qFx -- "$1"
+}
+
+# @ : list of items to be merged
+# @desc : very inefficient way for ensuring uniqueness without sorting
+function __merge_array
+{
+ cat -n <(printf "%s\n" "${@}") | sort -t$'\t' -k2,2 -u | \
+ sort -t$'\t' -k1,1 | cut -d$'\t' -f 2
+}
+
+function __append_item
+{
+ # emulating `+=` which adds items to the end of the list
+ local -ar __tmp_a=( "${@:2}" "$1" )
+ __merge_array "${__tmp_a[@]}"
+}
+
+function is_in_options
+{
+ __is_in_array "$1" ${!OPTIONS[@]}
+}
+
+function is_in_modules
+{
+ __is_in_array "$1" ${MODULES[@]}
+}
+
+function are_in_modules
+{
+ local i
+ for i in "${@}"; do __is_in_array "$i" "${MODULES[@]}" || return $s_err; done
+}
+
+function is_in_optarg
+{
+ __is_in_array "$1" ${OPTARG[@]}
+}
+
+function module_enable
+{
+ MODULES=( $(__append_item "$1" "${MODULES[@]}") )
+}
+
+function getflag
+{
+ local -ar flags=( ${2//:/ } )
+
+ __is_in_array "$1" ${flags[@]} && echo "true" || echo "false"
+}
+
+function setflag
+{
+ echo "${*// /:}"
+}
diff --git a/util/validation.sh b/util/validation.sh
new file mode 100644
index 0000000..4b5f206
--- /dev/null
+++ b/util/validation.sh
@@ -0,0 +1,1057 @@
+#!/bin/bash
+#
+# validation.sh
+#
+# Copyright 2019 Luigi Santivetti <luigi.santivetti@gmail.com>
+
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# @1 : holder variable name - lhs
+# @desc : if @1 matches against $rex_legal_holder_variable, then convert
+# : it back to the original module name
+function get_modname_from_variable
+{
+ is_valid_holder_lhs "${*}" || return $s_err
+
+ # Remove leading underscore, strip longest underscore match from end,
+ # finally lowercase it
+ local mod=$1; mod=${mod:1}; mod=${mod%%_*}; mod=${mod,,}
+
+ # Verify there is one
+ [ -d "$module_d/$mod" ] || {
+ lets -l -e "'${mod:-undefined-module}' not found"
+ return $s_err
+ }
+
+ echo $mod # Success
+}
+
+# @1 : absolute path to a module's sh file
+# @desc : derive module name from holder.sh path
+function get_modname_from_file
+{
+ local modname
+
+ [ -f "$1" ] && {
+ modname="$(dirname "$1")"; modname=${modname##*/}; echo $modname
+ } || echo ""
+}
+
+# @1 : holder variable name
+function is_valid_holder_lhs
+{
+ [ -n "$1" ] || return $s_err
+
+ # Is this variable a holder legal one
+ echo $1 | grep -qE -- "$rex_legal_holder_variable"
+
+ [ ${PIPESTATUS[1]} -eq 0 ]
+}
+
+# @1 : module name
+function is_valid_dependency
+{
+ local -r modname="$1"
+
+ if is_in_modules $modname; then
+ return $s_ok
+ elif is_in_optarg $modname; then
+ lets -l -w "'$modname' was requested, but it's turned off"
+ else
+ local m_list="${MODULES[*]},$modname"; m_list="${m_list// /,}"
+ lets -l -d "adding dependency: '$module' on '$modname'"
+
+ is_opt "check" && module_enable "$1" && return $s_ok
+
+ # we cannot source a new module within this subshell, they need to be
+ # included from the beginning.
+ is_opt "watch" "doins" "upins" "upall" "doall" && \
+ lets -l -e "$(get_opt) cannot complete, run --check ${MODULES[@]}"
+ fi
+
+ return $s_err
+}
+
+# @1 : module name
+function is_in_done
+{
+ local -r modname="$1"
+
+ __is_in_array "$modname" "${__done[@]}"
+}
+
+# @1 : module name
+function push_done_deps
+{
+ local -r modname="$1"
+
+ is_in_done "$modname" || __done+=( "$modname" )
+}
+
+# @1 : module name
+function is_in_deps
+{
+ local -r modname="$1"
+
+ __is_in_array "$modname" "${__deps[@]}"
+}
+
+# @1 : module name
+function pop_pending_deps
+{
+ local -r modname="$1"
+
+ if ! is_in_deps "$modname"; then
+ lets -l -e "dependency is missing: $modname"
+ return $s_err
+ fi
+
+ unset '__deps[-1]'
+}
+
+# @1 : module name
+function push_pending_deps
+{
+ local -r modname="$1"
+
+ if is_in_deps "$modname"; then
+ lets -l -e "circular dependecy on $modname"
+ return $s_err
+ fi
+
+ __deps+=( "$modname" )
+}
+
+# @1 : module name
+function update_dependecies
+{
+ local -r modname="$1"
+
+ pop_pending_deps $modname && push_done_deps $modname && \
+ return $s_ok || return $s_err
+
+ return $s_ok # __deps can be 0
+}
+
+# @1 : maybe unbound error from stderr
+function __is_valid_stderr_unbound
+{
+ local -r __error="unbound"
+
+ [ "$1" = "$__error" ] && return $s_ok
+
+ return $s_err
+}
+
+# @1 : maybe word from stderr
+function __is_valid_stderr_words
+{
+ [ "$(wc -w <<< "$*")" -eq 1 ] && return $s_ok
+
+ lets -l -e "'$*' unknown words"
+ return $s_err
+}
+
+# @1 : error message delimiter
+function __is_valid_stderr_delimeter
+{
+ local -r regex=":$"
+
+ (echo "$1" | grep -qE -- "$regex") && return $s_ok
+
+ lets -l -e "'$1' unknown delimiter"
+ return $s_err
+}
+
+# @1 : tail offset for unbound variable name in stderr message
+function is_valid_stderr_unbound
+{
+ local -ir offset=$1; shift
+ local -ar stderr=( ${@} )
+ local -ir unbofs="$offset - 1"
+
+ # Don't get crazy about validation, this is clearly fragile and it
+ # will just break on a different shell.
+
+ [ ${#stderr[@]} -ge $offset ] || {
+ lets -l -e "$offset is oob ${#stderr[@]}, ${stderr[*]}"
+ return $s_err
+ }
+
+ __is_valid_stderr_unbound ${stderr[-$unbofs]} || {
+ lets -l -e "stderr reported:"
+ lets -l -e "${stderr[@]}";
+ return $s_err
+ }
+
+ __is_valid_stderr_words ${stderr[-$offset]} && \
+ __is_valid_stderr_delimeter ${stderr[-$offset]} || return $s_err
+}
+
+# @1 : module name
+function get_module_complete
+{
+ local -r modname="$1"; shift
+
+ if [ ${#__deps[@]} -gt 0 ]; then
+ update_dependecies $modname && return $s_ok || return $s_err
+ fi
+
+ if [ ${#__deps[@]} -eq 0 ]; then
+ if is_opt "watch" "doins" "upins" "upall" "doall"; then
+ local regex="$rex_legal_dynamic_typedef"
+ regex+="[[:space:]]$rex_legal_holder_variable"
+ regex+="$rex_legal_holder_assignment_op\""
+
+ local closing_lhs
+ closing_lhs="$dynamic_typedef _${modname^^}_${script_now}_COUNT_"
+
+ local -i closing_rhs
+ closing_rhs="$(printf "%s\n" "${__envv[@]}" | grep -oE -- "$regex" | wc -l)"
+
+ [ ${#__envv[@]} -eq $closing_rhs ] || {
+ lets -l -e "variable count mismatch: ${#__envv[@]} vs $closing_rhs"
+ return $s_err
+ }
+
+ # Flag that $file_rhs is fully bound, accessing this as an array at -1
+ # always gives the module name of the environment.
+ echo "$closing_lhs=\"$closing_rhs\";"
+ elif is_opt "check"; then
+ echo "local -a deps=( ${__done[@]} )"
+ fi
+
+ # clean up
+ unset '__deps' && ! declare -p -- '__deps' &>/dev/null || return $s_err
+ unset '__done' && ! declare -p -- '__done' &>/dev/null || return $s_err
+ unset '__envv' && ! declare -p -- '__envv' &>/dev/null || return $s_err
+ unset '__ubnd' && ! declare -p -- '__ubnd' &>/dev/null || return $s_err
+ fi
+}
+
+# @ : list of lines read back from stdout
+function set_query_stdout_return_code
+{
+ # Uncoditionally print this on the stdout
+ printf "%s\n" "${@}"
+
+ if ([ -z "${*}" ] && [ -z "${__ubnd[$modname]}" ]); then
+
+ # Module fully bound, nothing to do
+ return $s_ok
+ elif ([ -n "${*}" ] && [ -n "${__ubnd[$modname]}" ]); then
+
+ # Unbound values fetched, so far so good
+ return $s_maybe_bound
+ elif ([ -n "${*}" ] && [ -z "${__ubnd[$modname]}" ]); then
+
+ # Unexpected stdout for this module
+ return $s_unk_stdout
+ elif ([ -z "${*}" ] && [ -n "${__ubnd[$modname]}" ]); then
+
+ # No values fetched for unbound variables
+ return $s_null
+ else
+
+ # It should not get here - bug
+ return $s_err2
+ fi
+}
+
+# @ : list of lines read back from stderr
+function set_query_stderr_return_code
+{
+ local -ar __lines=( "${@}" ); local -ir offset=3
+ local __l; local -a __la
+
+ for __l in "${__lines[@]}"; do
+
+ # Turn a line into an array of words
+ __la=( ${__l} )
+
+ # Bail if not unbound variable stderr (smt we don't care about)
+ is_valid_stderr_unbound $offset "${__la[@]}" || return $s_err
+
+ # Leave variable name on the stdout, will try to bind it
+ echo "${__la[-$offset]%:}"
+ done
+
+ return $s_unbound
+}
+# @ : list of files to test source with
+function __test_source_helper
+{
+ local __f; local set_flags=""
+
+ is_set "errexit" || set_flags="+e"
+ is_set "nounset" || set_flags="$set_flags +u"
+ is_set "posix" || set_flags="$set_flags +o posix"
+
+ set -ue
+ for __f in ${@}; do
+ case "$__f" in
+ "$module_d/"*"/$holder_sh" ) set -o posix ;;
+ "$module_d/"*"/$scheme_sh" ) set -o posix ;;
+ esac
+
+ # Don't kill the subshell, collect the stderr/stdout first
+ (source "$__f" || true)
+
+ # Source to kill if an error occurs. In subshell because we
+ # don't care about the actual contents of $__f here.
+ (source $__f &>/dev/null)
+ done
+
+ [ -n "$set_flags" ] && eval "set $set_flags" || true
+}
+
+# @ : list of files to test source with
+function test_source
+{
+ local line __l
+
+ (__test_source_helper "${@}" 2>&1) | while read -r __l; do
+ echo -e "$__l\n"
+ done
+
+ return "${PIPESTATUS[0]}"
+}
+
+# @1 : flags for sourcing modules
+# @2 : list of modules whose files need sourcing
+function __source_files_helper
+{
+ local -r include_scheme="$(getflag "-s" "$1")";
+ local -r multi_source="$(getflag "-m" "$1")"
+ local -ar modules="${@:2}"
+ local -a __files=( ); local -i __e
+ local __mod __f
+
+ if [ "true" = "$multi_source" ]; then
+ # Order in sourcing modules matters
+ for __mod in ${modules[@]}; do
+ # Order in sourcing files matters
+ __files+=( "$module_d/$__mod/$module_sh" )
+ __files+=( "$module_d/$__mod/$holder_sh" )
+ __files+=( "$module_d/$__mod/$scheme_sh" )
+
+ for __f in ${__files[@]}; do
+ test_source $__f; __e=$?
+ [ $__e -eq 0 ] || return $__e
+ source "$__f" &>/dev/null
+ done
+ done
+ else
+ local __var; __mod="${modules[0]}"
+
+ # First, source external variable needed for this module, if any
+ local -a ext_var=( "$(get_filter_out_deps $__mod "${__envv[@]}")" )
+
+ [ -n "${ext_var[*]}" ] && source <(echo "${ext_var[@]}")
+
+ # Second, does it need module.sh?
+ [ "$__mod" != "$module" ] && __files+=( "$module_d/$__mod/$module_sh" )
+
+ # Third, always holder.sh
+ __files+=( "$module_d/$__mod/$holder_sh" )
+
+ # Fourth, does it need scheme.sh?
+ [ "true" = "$include_scheme" ] && __files+=( "$module_d/$__mod/$scheme_sh" )
+
+ for __f in ${__files[@]}; do
+ test_source $__f; __e=$?
+ [ $__e -eq 0 ] || return $__e
+ source "$__f"
+ done
+
+ # Print on stdout the value of unbound variables for this module
+ for __var in ${__ubnd["$__mod"]}; do
+ set -ue
+
+ # Equivalent to ${!__var}, but will exit reporting unbound
+ # variable name in ${__var}.
+ eval "printf \"%q\\\n\" \"\$${__var}\""
+ done
+ fi
+}
+
+# @1 : flags for sourcing modules
+# @2 : list of modules whose files need sourcing
+function source_files
+{
+ local -r flags="$1"; local -ar modules="${@:2}"
+ local line __l
+
+ (__source_files_helper "$flags" "${modules[@]}" 2>&1) | while read -r __l; do
+ echo -e "$__l\n"
+ done
+
+ return "${PIPESTATUS[0]}"
+}
+
+# @1 : flags for sourcing modules
+# @2 : list of modules whose files need sourcing
+function get_stdout_source
+{
+ local -i __e; local -a __var; local old_ifs="$IFS"
+ local -r flags="$1"; local -ar modules="${@:2}"
+
+ IFS=$'\n' __var=( $(source_files "$flags" "${modules[@]}") )
+ __e=$?; IFS="$old_ifs"
+
+ case "$__e" in
+
+ # no error and no stderr, however there could be something
+ # else on the stdout.
+ $s_ok ) printf "%s\n" "${__var[@]}"; return $s_ok ;;
+
+ # some error, maybe unbound, stderr
+ * ) printf "%s\n" "${__var[@]}"; return $s_err ;;
+ esac
+}
+
+# @1 : flags for sourcing modules
+# @2 : list of modules whose files need sourcing
+function get_query_subshell
+{
+ local -r old_ifs="$IFS"; local -r flags="$1"; local -ar modules="${@:2}"
+ local -i __e; local -a stdout
+
+ IFS=$'\n' stdout=( $(get_stdout_source "$flags" "${modules[@]}") )
+ __e=$?; IFS="$old_ifs"
+
+ case "$__e" in
+ $s_err ) set_query_stderr_return_code "${stdout[@]}" ;;
+ $s_ok ) set_query_stdout_return_code "${stdout[@]}" ;;
+ esac
+}
+
+# @1 : module name
+# @ : list of legal holder variable assignments
+function get_filter_out_deps
+{
+ local regex="$dynamic_typedef"
+ regex+=" _${1^^}"
+ regex+="$rex_legal_holder_assignment_lhs"
+
+ printf "%s\n" "${@:2}" | grep -vE -- "$regex"
+}
+
+# @ : list of variables
+function get_filter_mods
+{
+ local -r regex="^${rex_legal_holder_token}_"
+ local -a mods
+
+ mods="$(printf "%s\n" "${@}" | grep -oE -- "$regex" | sort -u; \
+ return ${PIPESTATUS[1]})"
+ if [ $? -ne $s_ok ]; then
+ lets -l -e "invalid modules: unknown: ${@}"
+ return $s_unk_stderr
+ fi
+
+ mods="${mods[@]//_/}"; mods="${mods[@],,}"
+
+ # TODO: validate found modules against available modules
+# if ! __is_in_array $(get_all_modules); then
+# lets -l -e "invalid modules: not in MODULES"
+# return $s_unk_stderr
+# fi
+
+ printf "%s\n" "${mods[@]}"; true
+}
+
+# TODO: integrate this function into the validation framework
+# @1 : module name
+function is_valid_pending_ubnd
+{
+ local var
+
+ for var in ${__ubnd["$1"]}; do
+ ! declare -p -- "$var" &>/dev/null || {
+ lets -l -e "$1: $var defined, corrupted environment"
+ return $s_err
+ }
+ done
+}
+
+# @ : list of unbound variables
+function push_pending_ubnd
+{
+ local mod; local -a mods
+
+ mods=( $(get_filter_mods "${@}") )
+ [ $? -eq $s_ok ] || return $?
+
+ for mod in ${mods[@]}; do
+ local -a ubnd=( $(printf "%s\n" "${@}" | grep -E -- "^_${mod^^}_") )
+ __ubnd["$mod"]="$(__merge_array "${ubnd[@]}" "${__ubnd[$mod]}")"
+ [ $? -eq $s_ok ] || return $?
+ done
+}
+
+# @1 : flags for sourcing modules
+# @2 : list of modules whose files need sourcing
+function get_query_state
+{
+ local flags="$1"; local -ar modules="${@:2}"
+ local -i query; local -a umod=( )
+
+ umod=( "$(get_query_subshell "$flags" ${modules[@]})" ); query=$?
+
+ # Verbose output, comment this in for more debug, off by default
+ #lets -l -d "$modname: [query=$query]: ${umod:-NO UMOD}"
+
+ # Leave this on the stdout
+ printf "%s\n" "${umod[@]}"
+
+ case "$query" in
+ $s_maybe_bound ) return $s_maybe_bound ;;
+ $s_unk_stdout ) return $s_unk_stdout ;;
+ $s_unbound ) return $s_maybe_unbound ;;
+ $s_err2 ) return $s_err ;; # bug
+ $s_err ) return $s_unk_stderr ;;
+ $s_ok ) return $s_ok ;;
+ esac
+}
+
+# @1 : module name
+# @ : list of holder variable names
+function __maybe_unbound_helper
+{
+ local -r modname="$1"; shift
+ local -ar ulhs=( "${@}" )
+ local -a umods
+ local depmod
+ local -i __e
+
+ umods=( $(get_filter_mods ${ulhs[@]}) )
+ [ $? -eq $s_ok ] || return $?
+
+ if __is_in_array $modname ${umods[@]}; then
+
+ # This module has is own unbound variables
+ lets -l -e "$modname: unbound variables: ${ulhs[@]}"
+ return $s_err
+ fi
+
+ push_pending_ubnd "${ulhs[@]}"; __e=$?
+ if [ $__e -ne $s_ok ]; then
+ lets -l -e "$modname: failed to add unbound variables"
+ return $__e
+ fi
+
+ for depmod in ${umods[@]}; do
+ if ! is_valid_dependency $depmod; then
+ lets -l -d "$depmod: invalid dependency"
+ return $s_err
+ fi
+
+ if ! push_pending_deps $depmod; then
+ lets -l -w "$deplhs might be unbound"
+ return $s_err
+ fi
+
+ # Valid dependency, go for another ride
+ get_file_extarnal_references $depmod
+ done
+}
+
+# @1 : module name
+# @2 : list of values for each variable in __ubnd, IFS being '\n'
+function __maybe_bound_helper
+{
+ local -r modname="$1"; shift
+ local -ar urhs=( ${@} )
+ local -ar ulhs=( ${__ubnd[$modname]} )
+ local assignment=""
+ local -i idx
+
+ if [ ${#ulhs[@]} -ne ${#urhs[@]} ]; then
+
+ # Mismatch in number of lhs and rhs
+ lets -l -e "$modname: LHS ${#ulhs[@]} vs RHS ${#urhs[@]}"
+ lets -l -e "LHS: ${ulhs[@]}"
+ lets -l -e "RHS: ${urhs[@]}"
+ return $s_err
+ fi
+
+ for idx in ${!ulhs[@]}; do
+ local lhs="${ulhs[$idx]}"
+ local rhs="$(echo "${urhs[$idx]}")"
+
+ if [ -z "$rhs" ] || [ "$rhs" = "''" ]; then
+ lets -l -e "$lhs value is null"
+ return $s_null
+ fi
+
+ assignment="$dynamic_typedef $lhs=\"$rhs\";"
+ if is_opt "watch" "doins" "upins" "upall" "doall"; then
+
+ # Only leave it on the stdout for those options
+ echo "$assignment"
+ fi
+
+ __envv+=( "$assignment" )
+ done
+
+ # clear __ubnd
+ unset __ubnd["$modname"]
+}
+
+# @- : (optional) flag for including scheme.sh in the query
+# @1 : lhs variable name or null for querying any unbound variable
+# @2 : module name
+# @return : stdout, a list of variable assignments. Specifically, only those
+# : whose rhs-value is an external reference.
+function get_file_extarnal_references
+{
+ local sf; case "$1" in -s) sf="-s"; shift ;; *) ;; esac
+
+ # This will be unset on exit
+ declare -p -- '__deps' &>/dev/null || declare -ga __deps=( )
+ declare -p -- '__done' &>/dev/null || declare -ga __done=( )
+ declare -p -- '__envv' &>/dev/null || declare -ga __envv=( )
+ declare -p -- '__ubnd' &>/dev/null || declare -gA __ubnd=( )
+
+ local -a ret=( ); local -i state
+ local -r modname="$1"
+
+ ret="$(get_query_state "$sf" "$modname")"; state="$?"
+
+ # Verbose output, comment this in for more debug, off by default
+ #lets -l -d "$modname: [state=$state]: ${ret[@]:-NO RET}"
+
+ # Update state with helpers work
+ case "$state" in
+ $s_maybe_unbound )
+ __maybe_unbound_helper $modname "${ret[@]}"; state=$?
+
+ # Return only if *not* $s_ok
+ [ $state -eq $s_ok ] || return $state ;;
+
+ $s_maybe_bound )
+ __maybe_bound_helper $modname "${ret[@]}"; state=$?
+
+ # Return only if *not* $s_ok
+ [ $state -eq $s_ok ] || return $state
+
+ # If $modname was done already, then early return
+ ! is_in_done $modname || {
+ pop_pending_deps $modname; state=$?
+
+ # Always return
+ return $state;
+ } ;;
+
+ $s_ok ) get_module_complete $modname; state=$?
+
+ # Return only if $s_ok
+ [ $state -eq $s_ok ] && return $s_ok ;;
+ esac
+
+ # Verbose output, comment this in for more debug, off by default
+ #lets -l -d "$modname: [state=$state]: UPD: ${ret[@]:-NO RET}"
+
+ case "$state" in
+ # Errors
+ $s_unk_stdout ) lets -l -e "unknown stdout" ; return $s_err ;;
+ $s_unk_stderr ) lets -l -e "unknown stderr" ; return $s_err ;;
+ $s_inv_lhs ) lets -l -e "invalid variable: $ret" ; return $s_err ;;
+ $s_unbound ) lets -l -e "unbound variable: $ret" ; return $s_err ;;
+ $s_null ) lets -l -e "$lhs value is null" ; return $s_err ;;
+ $s_err ) lets -l -e "query failed" ; return $s_err ;;
+ esac
+
+ [ $state -eq $s_ok ] && get_file_extarnal_references $sf $modname
+}
+
+# @1 : holder variable rhs assignment, missing leading double quote sign
+function is_valid_eval_rhs
+{
+ # Eval black magic
+ #
+ # RHS didn't pass regex validation, this can be for two reasons, a good one
+ # and a bad one. For a good reason, the line is of the form:
+ #
+ # LHS="RHS"; LHS2="RHS2"
+ # LHS="RHS"; for i in "${a[@]}"
+ # LHS="RHS" && LHS2="RHS2"
+ #
+ # these are only some examples of genuine failure, holder.sh needs fixing.
+ # For the a bad reason, the line is of the form:
+ #
+ # LHS="RHS$(command "")"
+ #
+ # An example of this is command substitution in the assignment RHS, this is
+ # a false positive that must be granted and pass validation.
+ #
+ # Regex RHS validation only allows one pair of unescaped double quotes per
+ # RHS, they are supposed to enclose the whole RHS from start to end, and any
+ # other double quote appearing within this region should be escaped.
+ #
+ # In the latter case above, regex validation detects text after the second
+ # encountered, unescaped, double quote. For instance from an input such as
+ # `"RHS$(command "arg")"`, it will match `arg")"` as illegal.
+ #
+ # Naturally bash doesn't demand an argument passed to a command substitution
+ # to be quoted, but here holder.sh files run with the errexit flag, thus one
+ # command might fail when provided with a null argument. On the other hand,
+ # such command could handle arguments of length 0 but not null, such as `""`,
+ # hence quoting every argument in holder files becomes necessary.
+ #
+ # This whole validation system is artificial, arbitrarily made by the author
+ # to make assumptions over the state of a template. It also aims to give the
+ # developer enough flexibility in writing holder files, for, not preventing
+ # him or her to make use of constructs like command substitution in holder.sh
+ # variables.
+ #
+ # To fix this, extend RHS validation when it fails and process further the
+ # supposedly illegal line. is_valid_eval_rhs() takes the RHS text that has
+ # matched as illegal against $rex_illegal_holder_assignment_rhs and it tries
+ # to establish if this is a false positive by using the built in `set -x`.
+ #
+ # When it gets to this point errexit already passed, so it is fair to assume
+ # this is a bash legal line.
+ (
+ local -i __stack_ops
+ local __stack_lvl
+ local __xline
+
+ while read -r __xline; do
+ if echo $__xline | grep -q -- "__stack_${script_now}"; then
+ __stack_lvl="$(echo $__xline | cut -d' ' -f 1)"
+ __stack_lvl="${__stack_lvl%%__stack_$script_now*}"
+ else
+ [ -z "${__xline%%$__stack_lvl *}" ] && ((__stack_ops++))
+ echo $__xline | grep -q -- "__lhs_${script_now}" && {
+ local __lhs_lvl="${__xline%% __lhs_$script_now*}"
+ [ "$__stack_lvl" = "$__lhs_lvl" ] || return $s_err2
+ }
+ fi
+ done < <(
+ eval "set -x; __stack_${script_now}=0; __lhs_${script_now}=\"$(echo $*)" 2>&1
+ ); wait $!
+
+ [ $__stack_ops -eq 1 ] 2>/dev/null || return $s_err
+ ) && return $s_ok
+
+ case "$?" in
+ $s_err ) lets -l -e "too many commands detected per line" ;;
+ $s_err2 ) lets -l -e "cannot determine commands per line" ;;
+ esac; return $s_err
+}
+
+# @1 : module name
+# @2 : holder.sh line
+# @desc : This function can only work with bash script that is known to be
+# : valid, as in, it makes assumptions upon valid bash syntex. Make sure
+# : the line given in $2 belongs to a valid bash script.
+function is_compliant_holder_line
+{
+ local modname=$1
+ local line=$2
+
+ case "$line" in
+
+ # Allow conditional statement
+ \if* | \else* | \elif* | \fi* ) return $s_ok ;;
+
+ # Allow blank lines
+ '\n' | '' ) return $s_ok ;;
+
+ # Allow comments
+ '#'* ) return $s_ok ;;
+ esac
+
+ # From this point, $line can only be an assignment
+ # Assignment validation, part 1: lhs
+ #
+ # $rex_legal_holder_assignment_lhs is a module generic pattern, so make it
+ # specific to this particular module. Also, $modname must be upper case.
+ local rex_legal_lhs="^_${modname^^}$rex_legal_holder_assignment_lhs"
+ local legal_lhs="$(echo "$line" | grep -oE -- $rex_legal_lhs)"
+ if [ -n "$legal_lhs" ]; then
+ local rhs=${line##$legal_lhs}
+
+ # Assignment validation, part 2: rhs
+ #
+ # At this point, according to $rex_illegal_holder_assignment_rhs,
+ # nothing is allowed on this line other than the assignment rhs.
+ local rex_illegal_rhs="$rex_illegal_holder_assignment_rhs"
+ local illegal_rhs="$(echo "$rhs" | grep -oE -- $rex_illegal_rhs)"
+ [ -z "$illegal_rhs" ] && return $s_ok
+
+ is_valid_eval_rhs "$rhs" && {
+ [ "${illegal_rhs: -1}" != '"' ] && \
+ lets -l -w "suspicious: $illegal_rhs"
+ return $s_ok # no error
+ }
+
+ lets -l -e "illegal RHS: $rhs"
+ else
+ lets -l -e "illegal LHS: $line"
+ fi
+
+ # Here is error only
+ return $s_err
+}
+
+# @1 : module name
+# @2 : absolute path to holder file
+function is_compliant_holder
+{
+ local modname=$1
+ local -i nline
+ local line
+
+ # Is it of the desired form?
+ while read -r line; do
+ ((nline++))
+ if ! is_compliant_holder_line $modname "$(echo "$line")"; then
+ lets -l -e "${2##$module_d/}: line $nline is not compliant"
+ return $s_err
+ fi
+ done < $2
+}
+
+# @1 : module name
+function is_compliant_module
+{
+ local -r modname=$1
+ local -r modfile="$module_d/$modname/$module_sh"
+ local -a valid_modname=""
+
+ (
+ read -a valid_modname < <(
+ # $module should never be set at this point
+ ! declare -p -- "module" &>/dev/null || return $s_err2
+
+ local -i err; local stdout
+ stdout="$(set -eu; source $modfile)"; err=$?
+ case "$err" in
+ $s_ok )
+ [ -n "$stdout" ] && {
+ echo "$stdout"; return $s_err3
+ } || { set -eu; source "$modfile" || return $err4; } ;;
+ * ) return $err4 ;;
+ esac
+
+ [ "$module" = "$modname" ] || return $s_err5
+
+ [ -n "$module" ] && [ -d "$module_d/$module" ] || return $s_err6
+
+ declare -p -- "mod_module_d" >/dev/null || return $s_err7
+ declare -p -- "depmod" &>/dev/null || \
+ { set +eu; lets -l -w "$modname doesn't have depmod"; set -eu; }
+
+ is_in_modules $module || return $s_err8
+
+ set +eu; echo "${depmod[@]} $module" && return $s_ok
+ ); wait $!
+ case "$?" in
+ $s_err2 ) lets -l -e "corrupted environment" ;;
+ $s_err3 ) lets -l -e "$modname/$module_sh spams the stdout"
+ lets -l -e "this was found on the stdout: ${valid_modname[*]}" ;;
+ $s_err4 ) lets -l -e "failed to source $modfile" ;;
+ $s_err5 ) lets -l -e "$modfile doesn't define ${modname:-undefined}" ;;
+ $s_err6 ) lets -l -e "${modname:-undefined} doesn't exist" ;;
+ $s_err7 ) lets -l -e "is '$modname' sourcing common.sh" ;;
+ $s_err8 ) lets -l -w "disabled: '$modname'"; return $s_disabled ;;
+ $s_ok ) lets -l -i "enabled: '$modname'"
+ printf "%s\n" "${valid_modname[@]}"; return $s_ok ;;
+ esac; return $s_err
+ ); return $?
+}
+
+# @1 : absolute path to bash script
+function run_bash_errexit
+{
+ { /bin/bash -o errexit $1 ${@:2} 2>/dev/null; } || {
+ lets -l -e "${1##$module_d/}: doesn't even source, errexit"
+ return $s_err
+ }
+}
+
+# @1 : absolute path to holder.sh
+# @desc : For efficiency and for semantic reasons, this function needs
+# : to do multiple things at once, otherwise it would and should
+# : have been broken down. Do:
+# :
+# : 1. validate holder.sh for errexit
+# : 2. validate module.sh for errexit and nounset
+# : 3. validate holder.sh syntax line by line
+# :
+# : Only if this function succeeds, any attempt to bind unbound
+# : variables in holders.sh is really meaningful.
+function __is_valid_module_static
+{
+ local modname="$(get_modname_from_file $1)"
+ local stdout; local -i ret; local -a deps
+
+ # Can holder run?
+ stdout="$(run_bash_errexit $1)"
+ case "$?" in
+ $s_ok ) [ -n "$stdout" ] && {
+ lets -l -w "${1##$module_d/} spams the stdout"
+ } ;;
+ * ) return $s_err ;;
+ esac
+
+ deps=( $(is_compliant_module $modname) ); ret=$?
+ is_compliant_holder $modname $1 || {
+ # If this is disabled maybe that's fine even if broken
+ [ $ret -eq $s_disabled ] && {
+ lets -l -w "$modname broken, can break the dependencies"
+ return $s_off_broken
+ } || return $s_err
+ }
+
+ [ $ret -eq $s_disabled ] && return $s_off_usable || {
+ printf "%s\n" "${deps[@]}"
+ return $ret
+ }
+}
+
+# @ : list of absolute path to holder.sh files
+# @return : error if at least one fails
+function is_valid_module
+{
+ local -a deps
+ local modname
+
+ lets -l -d "checking ${#} module(s): ${@}"
+ for modname in ${@}; do
+ local holder="$module_d/$modname/$holder_sh"
+ deps=$(__is_valid_module_static $holder)
+ case "$?" in
+ $s_off_broken ) [ ${#} -eq 1 ] && return $s_off_broken || continue ;;
+ $s_off_usable ) [ ${#} -eq 1 ] && return $s_off_usable || continue ;;
+ $s_ok ) MODULES+=( $modname )
+ MODULES=( $(__merge_array "${MODULES[@]}" "${deps[@]}") ) ;;
+ $s_err ) return $s_err ;;
+ esac
+ done
+}
+
+# @ : list of legal holder assignments
+function is_valid_env
+{
+ local -r modname=$1
+
+ # Returned count of variables set in the environment
+ #
+ # get_file_extarnal_references() returns the number of variables used in a
+ # scheme.sh and holder.sh as last variable of the list. Such count does not
+ # include itself (this is appended to the list afterwards).
+ local -i count="$2"
+
+ [ -n "${modname}" ] && shift 2 || {
+ lets -l -e "dynamic env not valid, missing modname"
+ return $s_err
+ }
+
+ # keep it simple, we know that's what it is called
+ local -r var="_${modname^^}_${script_now}_COUNT_"
+ local -ar module_env=( "${@}" )
+ (
+ # First, if the subshell is corrupted, bail out
+ unset $var && ! declare -p -- $var 2>/dev/null || return $s_err3
+
+ # Second, if get_valid_modulenv_dynamic() count doesn't match with
+ # the lenght of assignment list passed in, bail out.
+ [ $count -eq ${#module_env[@]} ] || return $s_err4
+
+ # Third, if the count done by get_file_extarnal_references() doesn't
+ # match $count - 1, bail out.
+ eval "${module_env[-1]}"
+ ((count--)); [ ${!var} -eq $count ] || return $s_err5
+ ) || {
+ case "$?" in
+ $s_err3 ) lets -l -e "corrupted environment" ;;
+ $s_err4 ) lets -l -e "list count mismatch: $count vs ${#module_env[@]}" ;;
+ $s_err5 ) lets -l -e "$modname: invalid: ${#module_env[@]} vs ${!var}" ;;
+ esac; return $s_err
+ }
+
+ lets -l -i "'$modname' bound with $(($count-1)) external references"; return $s_ok
+}
+
+# @ : list of module names
+# @return : stdout on success, a valid associative array declaration where every
+# : key is the module name and its value is a list of valid holder
+# : variable assignments. Note, this function take a list of modules, but
+# : as per current design it is always called provinding only one module.
+function get_module_dependencies
+{
+ local -A modulenv; local modname; local -a env=( )
+ local table="$1"; shift
+
+ lets -l -i "resolving ${#} module(s): ${@}"
+ for modname in ${@}; do
+ lets -l -d "working on '$modname'"
+ local -i psfd; local -i pspid
+
+ # Open a read fd, start processing a scheme file
+ exec {psfd}< <(
+ get_file_extarnal_references -s $modname
+ ); pspid=$!
+
+ if is_opt "watch" "upins" "upall" "doins" "doall"; then
+ local assignment; local -i count=0;
+
+ while read -u $psfd -r assignment; do
+
+ # A little validation of what is coming back from stdout, lhs only
+ local regex="$rex_legal_dynamic_assignment_lhs.*"
+ echo "$assignment" | grep -qE -- "$regex" || {
+ lets -l -e "invalid assignment: $assignment"
+
+ # Close fd, kill producer, wait and return an error
+ exec {psfd}<&-; kill $pspid; wait; return $s_err
+ }
+ env+=( "$assignment" ); ((count++))
+ done
+
+ wait $pspid && is_valid_env $modname $count "${env[@]}" || {
+ lets -l -e "couldn't resolve $modname"
+ return $s_err
+ }
+
+ elif is_opt "check"; then
+ wait $pspid; read -u $psfd -a env
+ [ $? -eq 0 ] || return $s_err
+ fi
+
+ # Collect readings
+ modulenv[$modname]+="${env[*]}"
+ exec {psfd}<&-
+ done
+
+ # Once processed all modules
+ [ -n "$table" ] && {
+ # return all valid environments and then make it readonly
+ local def="$(declare -p -- modulenv)"
+ (echo "${def/declare -A modulenv/declare -rA $table}")
+ return $s_ok # success
+ }
+
+ lets -l -e "cannot export module environment: no table"
+ return $s_err # misuse
+}