.. _timeline:
+.. currentmodule:: moe.eval
+
Evaluation timeline
====================
-Eval.init
----------
+`Eval.init`
+------------
-`Eval.init` is run before the main pipeline, does the following:
+:func:`Eval.init` is run before the main pipeline and does the following:
======= ==========
Time Action
\ Fix `CONFIG` and load the config file `CONFIG`
\ Fix `LOG`, `USER_LOG`, `VERBOSE`, `HOME`, `DEBUG_LEVEL`, `TDIR`
\ Start logging system (to `LOG` and `USER_LOG`)
-\ Check basic settings, directories and rights (currently checking only `TDIR`)
-\ Insert predefined hooks ([m]) and `HOOKS` to main pipeline
+\ Insert predefined hooks ([m]) and custom `HOOKS` to main pipeline
======= ==========
Main pipeline
-------------
-Ran by `Eval.run`.
+Ran by :func:`Eval.run`.
======= ==========
Time Action
======= ==========
-10 [m] Task **must** be ready (rsynced, pulled). Locate and fix `PDIR`
-15 [m] Load `TASK_CONFIG_FILE` (dep. on `PDIR`)
-20 [m] Check `TASK_TYPE` and depending on that, initialize the module and insert future actions
+5 [m :func:`Eval.hook_init_dirs`] Check basic settings and directories
+10 *Task must be ready (rsynced, pulled)*
+15 [m :func:`Eval.hook_load_task_config`] Load task config file
+20 [m :func:`Eval.hook_init_tasktype`] Import and initialize the task-type module
**For 'standard':**
------------------
25 [tt] Locate the source
30 [tt] Initialize sandbox module (if needed)
40 [tt] Initialize judge module, insert its hooks
60 [tt] Run test pipeline for each `TEST` of `TESTS`
-**Recovery**
+**Cleanup**
------------------
70 Any failure jumps to this time
-80 [*] Cleanup must be finished
-\ Write metafile
+80 Cleanup *must* be finished
+90 [m :func:`Eval.hook_write_metadata`] Write metafile
======= ==========
Test pipeline
20 [tt] Locate the data file, copy to
40 [tt] Run judge (inside sandbox)
50 [tt] Read/parse judge result
-**Recovery**
+**Cleanup**
--------------------------------
70 Any failure jumps to this time
80 [*] Cleanup must be finished