16 Moe and the derived modules use many configuration variables indicating i.e. working directories,
17 task name, compiler commandline and options, current test name and name of test file to use.
18 All values are treated as text and may contain unicode characters (although this is usually not advisable).
19 A variable may be either set to a simple fixed string or to a string containing expansions of other variables.
20 The configuration library also supports appending text, conditionals and a simple variable hierarchy.
22 For the complete configuration syntax grammar, see :mod:`moe.config_parser` documentation.
24 Generally, both whitespace and indentation are optional and do not matter, individual
25 operations must be separated by either ``;`` or newline, ``#`` introduces a comment until the end of the line.
27 In *moe*, the config files are assumed to be in UTF-8, but using non-ASCII characters are discouraged outside
28 comments (mainly because not all unix comands handle unicode well).
30 Variables and substitution
31 ^^^^^^^^^^^^^^^^^^^^^^^^^^
33 The names of the variables may consist of letters, numbers, underscore, dash and a dot. Dot acts as
34 a separator of the variable name hierarchy and may not occur at the beginning or the end of variable name,
35 nor adjacent to another dot. The tree hierarchy is provided only for user's convenience and does not play
36 any significant role in variable evaluation.
38 The value of a variable may be defined by several operations:
40 **SET**, ``VAR = "value"``
41 Sets the variable to the variable-expansion of the string.
43 **APPEND**, ``VAR += "value"``
44 Appends the variable-expansion of the string.
46 Value is a ``"``-delimited string that may contain substitutions in the form ``{VARNAME}`` i.e.
47 ``"source-{VERSION}.{SUFFIX}"``, where ``VERSION`` and ``SUFFIX`` are the substituted variables.
48 The string may contain any characters including unicode (although this is not always advisable).
49 Characters ``"``, ``{``, ``{`` and ``\`` have to be escaped with ``\``, i.e. ``\"``, ``\{``, ``\{`` and ``\\``.
51 When evaluating a variable, all the substitutions are evaluated as well. Every variable is evaluated
52 in the same way, independently of the priority of the operations it occurs in. The substitutions must
53 therefore be acyclic and a variable may not contain itself as a substitution.
59 Each operation has a priority usually depending on its origin, i.e. builtins, global config file, task config file
60 or commandline overrides.
61 Only the **SET** operation with the highest priority and higher priority **APPENDs** are used (as if all operations were
62 applied from the least priority operation to the highest).
63 Operations of the same priority are processed by their definition order (first to last).
65 For example, in builtins (priority 0) we may have::
67 COMPILE = "{CC} {OPTS} {INFILE} -o {OUTFILE}"
71 In config file (priority 30)::
73 OPTS += " -W{WARNOPT}" # OPTS is now "-O{OPTIMIZE} -W{WARNOPT}"
77 And in command-line overrides (priority 100)::
79 COMPILE = "cp {INFILE} {OUTFILE}"
81 Moe would then later set ``INFILE`` and ``OUTFILE`` at some point before evaluating and executing
82 the compilation command.
84 With these *lazy* semantics, it is possible to set a variable to a value depending
85 on both higher- and lower-priority operations.
88 The implementation is actually very efficient and recalculates only the necessary values,
89 caching the results. Adding/removing operations to variables only recursively invalidates the
90 (potentially) influenced cached values, these are recalculated lazily on-demand.
96 The variables form a tree-like hierarchy with levels separated by dots. The variables
97 may be defined either by providing the full name or by inside subtrees.
98 Subtrees provide all the *defined* variables with given dot-separated prefix.
99 Substituted variable names must be given by their full names. ::
104 # Defines LOG.VERBOSE
105 TEST.LOGFILE = "{LOG.DIR}/test.log"
106 # Defines LOG.TEST.LOGFILE
108 LOG.TEST { VERBOSE = "Y" }
109 # The subtree name may contain multi-level names
114 Any block of operations may be guarded by a condition. The conditions may be nested and they
115 consist of expressions containing ``and``, ``or``, ``not`` and variable/string (in)equalities. The strings
116 may contain substitutions.
118 The syntax is the following::
120 if (NAME == "{TASK}.cc") or (NAME == "{TASK}.cpp") or
121 ((LANG_HINT == "C") and (not FOO != "BAR"))
126 The curly brackets delimiting the block, as well as the parentheses in the boolean expression,
127 are mandatory. ``else`` is not supported. ``if VAR=="FOO" {...}`` is equivalent to ``if "{VAR}"=="FOO" {...}``.
133 .. highlight:: python
134 .. automodule:: moe.config
139 .. autoexception:: ConfigError
140 .. autoexception:: UndefinedError
141 .. autoexception:: VariableNameError
142 .. autoexception:: VariableFixedError
143 .. autoexception:: CyclicConfigError
148 The configuration environment :class:`ConfigTree` is implemented as a flat dictionary of variables
149 referenced by their full name.
151 .. autoclass:: ConfigTree
157 Every variable object :class:`ConfigVar` stores the list of its defining operations, the current
158 cached value and both the variables its current value depends on and the variables that currently depend on it.
159 All the dependencies are calculated upon evaluation and removed upon invalidation.
160 Only the operations and conditions influencing the current value contribute to dependencies.
162 .. note:: If no **SET** applies, a variable is still undefined even if some **APPEND** applies. This *might* change.
164 .. autoclass:: ConfigVar
170 Every operation object is either **SET** or **APPEND** and may be guarded by a
173 .. autoclass:: Operation
178 Every condition is defined by a formula, the result is cached in the same way as with :class:`ConfigVar`, including
179 dependency tracking. A condition may depend on another parent condition in case this condition is
180 nested inside another. In this case the parent condition is checked first and the parent condition is a dependency.
182 .. autoclass:: ConfigCondition
188 These classes should not be used directly.
190 .. autoclass:: ConfigElem
193 .. autoclass:: ConfigExpression
197 ----------------------------------------
199 ----------------------------------------
201 .. automodule:: moe.config_parser
203 .. autoclass:: ConfigSyntaxError
205 .. autofunction:: config_escape