X-Git-Url: http://mj.ucw.cz/gitweb/?a=blobdiff_plain;f=t%2Fdoc%2Fconfig.rst;h=a07ec96d75e0b77d1ae81d59ec29a4bdfbd8eab5;hb=976c301650130b3c6c7aa0df83e8c29ea2ffb650;hp=fe4066203a8e4ebefd45470e420c77089697cc52;hpb=750990d7dbbaf2740f290a26dab527ce4444cbfb;p=moe.git diff --git a/t/doc/config.rst b/t/doc/config.rst index fe40662..a07ec96 100644 --- a/t/doc/config.rst +++ b/t/doc/config.rst @@ -1,53 +1,205 @@ .. _config: +.. highlight:: none + +================= Moe configuration ================= +.. contents:: + :local: -.. contents:: +------------------- +Configuration logic +------------------- -The configuration consists of lazy definition-style configuration resolver and configuration parser. +Moe and the derived modules use many configuration variables indicating i.e. working directories, +task name, compiler commandline and options, current test name and name of test file to use. +All values are treated as text and may contain unicode characters (although this is usually not advisable). +A variable may be either set to a simple fixed string or to a string containing expansions of other variables. +The configuration library also supports appending text, conditionals and a simple variable hierarchy. -The configuration variables form a dot-separated tree, but internally the list of variables is just -flat with full names. +For the complete configuration syntax grammar, see :mod:`moe.config_parser` documentation. -Configuration tree -++++++++++++++++++ +Generally, both whitespace and indentation are optional and do not matter, individual +operations must be separated by either ``;`` or newline, ``#`` introduces a comment until the end of the line. + +In *moe*, the config files are assumed to be in UTF-8, but using non-ASCII characters are discouraged outside +comments (mainly because not all unix comands handle unicode well). + +Variables and substitution +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The names of the variables may consist of letters, numbers, underscore, dash and a dot. Dot acts as +a separator of the variable name hierarchy and may not occur at the beginning or the end of variable name, +nor adjacent to another dot. The tree hierarchy is provided only for user's convenience and does not play +any significant role in variable evaluation. + +The value of a variable may be defined by several operations: + +**SET**, ``VAR = "value"`` + Sets the variable to the variable-expansion of the string. + +**APPEND**, ``VAR += "value"`` + Appends the variable-expansion of the string. + +Value is a ``"``-delimited string that may contain substitutions in the form ``{VARNAME}`` i.e. +``"source-{VERSION}.{SUFFIX}"``, where ``VERSION`` and ``SUFFIX`` are the substituted variables. +The string may contain any characters including unicode (although this is not always advisable). +Characters ``"``, ``{``, ``{`` and ``\`` have to be escaped with ``\``, i.e. ``\"``, ``\{``, ``\{`` and ``\\``. + +When evaluating a variable, all the substitutions are evaluated as well. Every variable is evaluated +in the same way, independently of the priority of the operations it occurs in. The substitutions must +therefore be acyclic and a variable may not contain itself as a substitution. + + +Operation priority +^^^^^^^^^^^^^^^^^^ + +Each operation has a priority usually depending on its origin, i.e. builtins, global config file, task config file +or commandline overrides. +Only the **SET** operation with the highest priority and higher priority **APPENDs** are used (as if all operations were +applied from the least priority operation to the highest). +Operations of the same priority are processed by their definition order (first to last). + +For example, in builtins (priority 0) we may have:: + + COMPILE = "{CC} {OPTS} {INFILE} -o {OUTFILE}" + OPTS = "-O{OPTIMIZE}" + CC = "gcc" + +In config file (priority 30):: + + OPTS += " -W{WARNOPT}" # OPTS is now "-O{OPTIMIZE} -W{WARNOPT}" + WARNOPT = "all" + OPTIMIZE = "0" + +And in command-line overrides (priority 100):: + + COMPILE = "cp {INFILE} {OUTFILE}" + +Moe would then later set ``INFILE`` and ``OUTFILE`` at some point before evaluating and executing +the compilation command. + +With these *lazy* semantics, it is possible to set a variable to a value depending +on both higher- and lower-priority operations. + +.. note:: + The implementation is actually very efficient and recalculates only the necessary values, + caching the results. Adding/removing operations to variables only recursively invalidates the + (potentially) influenced cached values, these are recalculated lazily on-demand. + + +Variables hierarchy +^^^^^^^^^^^^^^^^^^^ +The variables form a tree-like hierarchy with levels separated by dots. The variables +may be defined either by providing the full name or by inside subtrees. +Subtrees provide all the *defined* variables with given dot-separated prefix. +Substituted variable names must be given by their full names. :: + + LOG.DIR = "./log" + LOG { + VERBOSE = "N" + # Defines LOG.VERBOSE + TEST.LOGFILE = "{LOG.DIR}/test.log" + # Defines LOG.TEST.LOGFILE + } + LOG.TEST { VERBOSE = "Y" } + # The subtree name may contain several level names + +Conditionals +^^^^^^^^^^^^ + +Any block of operations may be guarded by a condition. The conditions may be nested and +consist of expressions containing ``and``, ``or``, ``not`` and variable/string (in)equalities. The strings +may contain substitutions. + +The syntax is the following:: + + if (NAME == "{TASK}.cc") or (NAME == "{TASK}.cpp") or + ((LANG_HINT == "C") and (not FOO != "BAR")) + { + LANG = "C" + } + +The curly brackets delimiting the bloc, as well as the brackets in the boolena expression, +are mandatory. ``else`` is not supported. ``if VAR=="FOO" {...}`` is equivalent to ``if "{VAR}"=="FOO" {...}``. + + +------------------- +Module `moe.config` +------------------- +.. highlight:: python .. automodule:: moe.config - Exceptions - ---------- +Exceptions +^^^^^^^^^^ + +.. autoexception:: ConfigError +.. autoexception:: UndefinedError +.. autoexception:: VariableNameError +.. autoexception:: VariableFixedError +.. autoexception:: CyclicConfigError + +Configuration tree +^^^^^^^^^^^^^^^^^^ + +The configuration environment :class:`ConfigTree` is implemented as a flat dictionary of variables +referenced by their full name. + +.. autoclass:: ConfigTree + :members: - .. autoclass:: ConfigError - .. autoclass:: UndefinedError - .. autoclass:: VariableNameError - .. autoclass:: VariableFixedError - .. autoclass:: CyclicConfigError +Variables +^^^^^^^^^ - Configuration tree - ------------------ +Every variable object :class:`ConfigVar` stores the list of its defining operations, the current +cached value and both the variables its current value depends on and the variables that currently depend on it. +All the dependencies are calculated upon evaluation and removed upon invalidation. +Only the operations and conditions influencing the current value contribute to dependencies. - .. autoclass:: ConfigTree - :members: +.. note:: If no **SET** applies, a variable is still undefined even if some **APPEND** applies. This *might* change. - Main config elements - -------------------- +.. autoclass:: ConfigVar + :members: - .. autoclass:: ConfigElem - :members: - .. autoclass:: ConfigCondition - :members: - .. autoclass:: ConfigVar - :members: - - Internal config elements - ------------------------ +Operations +^^^^^^^^^^ - .. autoclass:: Operation - .. autoclass:: ConfigExpression +Every operation object is either **SET** or **APPEND** and may be guarded by a +condition. -Config parser -+++++++++++++ +.. autoclass:: Operation + +Conditions +^^^^^^^^^^ + +Every condition is defined by a formula, the result is cached in the same way as with :class:`ConfigVar`, including +dependency tracking. A condition may depend on another parent condition in case this condition is +nested inside another. In this case the parent condition is checked first and the parent condition is a dependency. + +.. autoclass:: ConfigCondition + :members: + +Internals +^^^^^^^^^ + +These classes should not be used directly. + +.. autoclass:: ConfigElem + :members: + +.. autoclass:: ConfigExpression + + + +---------------------------------------- +Configuration parser +---------------------------------------- .. automodule:: moe.config_parser + +.. autoclass:: ConfigSyntaxError + +.. autofunction:: config_escape