Moe configuration
=================
-.. contents::
- :local:
+.. contents::
+ :local:
-------------------
Configuration logic
-------------------
-Moe and the derived modules use many configuration variables indicating i.e. working directories,
-task name, compiler commandline and options, current test name and name of test file to use.
+Moe and the derived modules use many configuration variables indicating i.e. working directories,
+task name, compiler commandline and options, current test name and name of test file to use.
All values are treated as text and may contain unicode characters (although this is usually not advisable).
A variable may be either set to a simple fixed string or to a string containing expansions of other variables.
The configuration library also supports appending text, conditionals and a simple variable hierarchy.
For the complete configuration syntax grammar, see :mod:`moe.config_parser` documentation.
-Generally, both whitespace and indentation are optional and do not matter, individual
+Generally, both whitespace and indentation are optional and do not matter, individual
operations must be separated by either ``;`` or newline, ``#`` introduces a comment until the end of the line.
In *moe*, the config files are assumed to be in UTF-8, but using non-ASCII characters are discouraged outside
^^^^^^^^^^^^^^^^^^^^^^^^^^
The names of the variables may consist of letters, numbers, underscore, dash and a dot. Dot acts as
-a separator of the variable name hierarchy and may not occur at the beginning or the end of variable name,
+a separator of the variable name hierarchy and may not occur at the beginning or the end of variable name,
nor adjacent to another dot. The tree hierarchy is provided only for user's convenience and does not play
-any significant role in variable evaluation.
+any significant role in variable evaluation.
The value of a variable may be defined by several operations:
Appends the variable-expansion of the string.
Value is a ``"``-delimited string that may contain substitutions in the form ``{VARNAME}`` i.e.
-``"source-{VERSION}.{SUFFIX}"``, where ``VERSION`` and ``SUFFIX`` are the substituted variables.
-The string may contain any characters including unicode (although this is not always advisable).
+``"source-{VERSION}.{SUFFIX}"``, where ``VERSION`` and ``SUFFIX`` are the substituted variables.
+The string may contain any characters including unicode (although this is not always advisable).
Characters ``"``, ``{``, ``{`` and ``\`` have to be escaped with ``\``, i.e. ``\"``, ``\{``, ``\{`` and ``\\``.
-When evaluating a variable, all the substitutions are evaluated as well. Every variable is evaluated
-in the same way, independently of the priority of the operations it occurs in. The substitutions must
+When evaluating a variable, all the substitutions are evaluated as well. Every variable is evaluated
+in the same way, independently of the priority of the operations it occurs in. The substitutions must
therefore be acyclic and a variable may not contain itself as a substitution.
Operation priority
^^^^^^^^^^^^^^^^^^
-Each operation has a priority usually depending on its origin, i.e. builtins, global config file, task config file
+Each operation has a priority usually depending on its origin, i.e. builtins, global config file, task config file
or commandline overrides.
Only the **SET** operation with the highest priority and higher priority **APPENDs** are used (as if all operations were
-applied from the least priority operation to the highest).
+applied from the least priority operation to the highest).
Operations of the same priority are processed by their definition order (first to last).
For example, in builtins (priority 0) we may have::
COMPILE = "cp {INFILE} {OUTFILE}"
-Moe would then later set ``INFILE`` and ``OUTFILE`` at some point before evaluating and executing
+Moe would then later set ``INFILE`` and ``OUTFILE`` at some point before evaluating and executing
the compilation command.
With these *lazy* semantics, it is possible to set a variable to a value depending
-on both higher- and lower-priority operations.
+on both higher- and lower-priority operations.
-.. note::
- The implementation is actually very efficient and recalculates only the necessary values,
- caching the results. Adding/removing operations to variables only recursively invalidates the
+.. note::
+ The implementation is actually very efficient and recalculates only the necessary values,
+ caching the results. Adding/removing operations to variables only recursively invalidates the
(potentially) influenced cached values, these are recalculated lazily on-demand.
^^^^^^^^^^^^^^^^^^^
The variables form a tree-like hierarchy with levels separated by dots. The variables
-may be defined either by providing the full name or by inside subtrees.
+may be defined either by providing the full name or by inside subtrees.
Subtrees provide all the *defined* variables with given dot-separated prefix.
Substituted variable names must be given by their full names. ::
LOG.DIR = "./log"
LOG {
- VERBOSE = "N"
+ VERBOSE = "N"
# Defines LOG.VERBOSE
TEST.LOGFILE = "{LOG.DIR}/test.log"
- # Defines LOG.TEST.LOGFILE
+ # Defines LOG.TEST.LOGFILE
}
LOG.TEST { VERBOSE = "Y" }
# The subtree name may contain several level names
^^^^^^^^^^^^
Any block of operations may be guarded by a condition. The conditions may be nested and
-consist of expressions containing ``and``, ``or``, ``not`` and variable/string (in)equalities. The strings
+consist of expressions containing ``and``, ``or``, ``not`` and variable/string (in)equalities. The strings
may contain substitutions.
The syntax is the following::
- if (NAME == "{TASK}.cc") or (NAME == "{TASK}.cpp") or
+ if (NAME == "{TASK}.cc") or (NAME == "{TASK}.cpp") or
((LANG_HINT == "C") and (not FOO != "BAR"))
{
LANG = "C"
}
-The curly brackets delimiting the bloc, as well as the brackets in the boolena expression,
+The curly brackets delimiting the bloc, as well as the brackets in the boolena expression,
are mandatory. ``else`` is not supported. ``if VAR=="FOO" {...}`` is equivalent to ``if "{VAR}"=="FOO" {...}``.
Configuration tree
^^^^^^^^^^^^^^^^^^
-The configuration environment :class:`ConfigTree` is implemented as a flat dictionary of variables
-referenced by their full name.
+The configuration environment :class:`ConfigTree` is implemented as a flat dictionary of variables
+referenced by their full name.
.. autoclass:: ConfigTree
:members:
Operations
^^^^^^^^^^
-Every operation object is either **SET** or **APPEND** and may be guarded by a
+Every operation object is either **SET** or **APPEND** and may be guarded by a
condition.
.. autoclass:: Operation
^^^^^^^^^^
Every condition is defined by a formula, the result is cached in the same way as with :class:`ConfigVar`, including
-dependency tracking. A condition may depend on another parent condition in case this condition is
+dependency tracking. A condition may depend on another parent condition in case this condition is
nested inside another. In this case the parent condition is checked first and the parent condition is a dependency.
.. autoclass:: ConfigCondition
Moe status files
================
-.. contents::
- :local:
+.. contents::
+ :local:
-------------------
Status files syntax
The status files hold and transmit status information of a single task (submission).
A status is an (unordered) dictionary of **key:value** entries and named subtrees.
-Keys of both the entries and subtrees may consist only of letters, numbers, ``-`` and ``_``
+Keys of both the entries and subtrees may consist only of letters, numbers, ``-`` and ``_``
(must match ``[a-zA-Z0-9-_]+``). The keys are case-sensitive.
The values are arbitrary byte-strings not containing character ``\0`` (ASCII char. 0).
-Note that this permits almost any encoding that can avoid the character ``\0``, such as UTF-8.
+Note that this permits almost any encoding that can avoid the character ``\0``, such as UTF-8.
When processing the values as strings, Moe decodes the strings as UTF-8 (unless stated otherwise).
-The format is line oriented (delimited by ``\n``),
+The format is line oriented (delimited by ``\n``),
whitespace characters (`` \t\r``) at the beginning of a line are ignored.
-The file consists of empty lines, comments, entries and subtrees.
+The file consists of empty lines, comments, entries and subtrees.
-Comments start with ``#`` and continue until the end of the line.
+Comments start with ``#`` and continue until the end of the line.
Comments must occur alone on the line.
-Entries are written as a key and a colon. Everything after the colon (incl. whitespace)
+Entries are written as a key and a colon. Everything after the colon (incl. whitespace)
is considered to be part of the value.
Multi-line values continue on following lines, prefixed by a single semicolon.
A subtree consists of a key followed by ``(``, the subtree is closed by ``)`` on a single line.
-Keys of entries and subtrees must be unique in every subtree, including the top-level of a status file.
+Keys of entries and subtrees must be unique in every subtree, including the top-level of a status file.
Status file grammar
^^^^^^^^^^^^^^^^^^^
ELEMENT = WS '\n' | COMMENT | ENTRY | SUBTREE
SUBTREE = WS KEY '(' WS '\n' STATUS ')' WS '\n'
ENTRY = WS KEY WS ':' VALUE '\n' ( WS ':' VALUE '\n' )*
- COMMENT = WS re('#[^\n\0]*\n'
+ COMMENT = WS re('#[^\n\0]*\n'
KEY = re([a-zA-Z0-9-_]+)
VALUE = re([^\n\0]*)
WS = re(['\r\t ']*)
-Every ``ELEMENT`` ends with a newline. ``ENTRY`` is a (possibly multiline) **key:value** pair.
+Every ``ELEMENT`` ends with a newline. ``ENTRY`` is a (possibly multiline) **key:value** pair.
---------------------
Updating status files
projects / moe.git / commitdiff