Cleanups:
o Clean up data types and make sure they cannot overflow. (size_t vs. u64 vs. sh_off_t vs. uns)
+o Clean up log levels.
+o Clean up introductory comments.
o Log messages should show both original and new size of the data. The speed
should be probably calculated from the former.
-o Automatically tune ASORT_MIN_RADIX, ASORT_MIN_SHIFT and especially ASORT_RADIX_BITS.
-o Check undefs in sorter.h and array.h.
+o Buffer sizing in shep-export.
+o Problems with thread stack limit in radix-sorting of arrays.
-Improvements:
-o Switching between direct and normal I/O. Should use normal I/O if the input is small enough.
-o How does the speed of radix splitting decrease with increasing number of hash bits?
- Does it help to use more bits than we need, so that we sort less data in memory?
-o Add automatic joining to the custom presorter interface?
-
-Users of lib/arraysort.h on big arrays (consider conversion to lib/sorter/array.h):
-
-indexer/chewer.c fixed hash + others
-indexer/chewer.c u32 id
-indexer/imagesigs.c fixed s32
-indexer/lexfreq.c ptr indirect int
-indexer/lexorder.c ptr complex
-indexer/lexorder.c ptr complex
-indexer/lexsort.c ptr complex
-indexer/mergeimages.c fixed s32
-indexer/mkgraph.c u32 indirect int
-indexer/mkgraph.c 2*u32 complex, but have hash
-indexer/mkgraph.c 2*u32 complex, but have hash
-indexer/reftexts.c fixed indirect int
+Users of lib/sorter/array.h which might use radix-sorting:
+indexer/chewer.c
+indexer/lexfreq.c
+indexer/mkgraph.c
+indexer/reftexts.c