X-Git-Url: http://mj.ucw.cz/gitweb/?a=blobdiff_plain;f=lib%2Fsorter%2FTODO;h=c2b653d5b96f4b2e5a3861befdb1cbbeddc88cd8;hb=90afcc18dbf7cb6c682e1efb994007f03e304422;hp=679649929abe3ac86c64546c45ea1377ae9ef7a3;hpb=8d1296ef1224f72a0c6e875208da0048830d89a9;p=libucw.git diff --git a/lib/sorter/TODO b/lib/sorter/TODO index 67964992..c2b653d5 100644 --- a/lib/sorter/TODO +++ b/lib/sorter/TODO @@ -3,13 +3,22 @@ o Giant runs. o Records of odd lengths. o Empty files. -Improvements: -o Use radix-sort for internal sorting. -o Parallelization of internal sorting. +Cleanups: o Clean up data types and make sure they cannot overflow. (size_t vs. u64 vs. sh_off_t vs. uns) +o Clean up log levels. +o Clean up introductory comments. +o Log messages should show both original and new size of the data. The speed + should be probably calculated from the former. +o Automatically tune ASORT_MIN_RADIX, ASORT_MIN_SHIFT and especially ASORT_RADIX_BITS. +o Buffer sizing in shep-export. + +Improvements: o Switching between direct and normal I/O. Should use normal I/O if the input is small enough. -o Deal with too rough range estimates in radix splitting. o How does the speed of radix splitting decrease with increasing number of hash bits? Does it help to use more bits than we need, so that we sort less data in memory? -o Log messages should show both original and new size of the data. The speed - should be probably calculated from the former. + +Users of lib/sorter/array.h which might use radix-sorting: +indexer/chewer.c +indexer/lexfreq.c +indexer/mkgraph.c +indexer/reftexts.c