5.1. Structure

These are not “pure” shell scripts, but are preprocessed first by ecFlow. In general, they should be simple shell wrappers (ideally generated by pyFlow, but that is beyond the scope of this document) around a standalone script (or a short sequence of such scripts with minimal logic), passing through any ecFlow-variable-derived options via command line arguments. Where there are a large number of such variables, they may be placed in a configuration file in the task’s temporary working directory that is then passed as a single command-line argument.

Options that come not from ecFlow variables but directly from a suite-level (or sub-suite component-level) configuration file (e.g. current config.*.h or their successor) should be passed by one of the following, depending how many need to be supplied:

  1. individually as command-line arguments, if only a few are required;

  2. via a temporary configuration file as with ecFlow-derived variables, if there are too many to pass individually;

  3. by directly passing the entire higher-level configuration file, if a large portion of its content is required;

  4. by passing a coherent sub-portion of the higher-level configuration structure, if a hierarchical configuration system is introduced in future.

In very limited cases, but only where it corresponds to a well-established interface (e.g. specifying paths to certain tools), options may be used to set environment variables instead; however as these are effectively a form of global variable this should not be the norm, and should never be done for options that may vary between components of a suite. Exporting the entire content of config.*.h to the environment via set -a / -o allexport is strongly deprecated.

In all cases, the interface to the called script should be well documented, so that the script which actually does the work can be tested outside of ecFlow. This called script must correctly report any failure via a non-zero return code to the task wrapper.

In light of the above, only the header and footer of the task wrappers, or the boilerplate code directly %included in them, should include ecFlow substitutions (%VAR%, further %includes etc.) Called / sourced scripts should not include such syntax, as this makes them impossible to use or test outside of ecFlow. If they are deployed via a construct like this in the task wrapper:

cat >$TMPDIR/script <<\EOF
%includenopp <script>
EOF
chmod +x $TMPDIR/script

then this should always be done using %includenopp rather than %include to prevent such substitutions and the need to “double-up” real % characters in the script. Similarly, this should be done with <<\ rather than << to prevent shell substitutions during deployment (which should only happen at runtime). However, future alternative mechanisms for script deployment are possible but outside the scope of this document, to be considered alongside the wider evolution of workflow code for suite generation and deployment.