Martin Mares [Sat, 17 Feb 2007 21:02:21 +0000 (22:02 +0100)]
Replaced the old timing functions by something more useful.
get_timer() using global state was making it unusable in many cases,
especially in library routines. I decided to make the callers specify
an explicit context (a timer) instead and re-used the idea of millisecond
timestamps from mainloop.c.
I've moved timestamp_t to config.h, introduced global get_timestamp()
and replaced the timer functions by ones with an explicit timestamp_t *
argument.
Martin Mares [Fri, 2 Feb 2007 22:27:02 +0000 (23:27 +0100)]
Implemented SORT_UNIFY, SORT_UNIQUE and debugged SORT_VAR_DATA.
Now the new sorter is equivalent to the old sorter (i.e., the same algorithms),
but with new interface and standing on new foundations. I will write
a simple test suite and then start filling the gaps with something
interesting.
Martin Mares [Fri, 2 Feb 2007 21:28:43 +0000 (22:28 +0100)]
Implemented swapping in/out buckets.
In some sorting methods, we want to use lots of buckets, but avoid
keeping all the files open and their buffers allocated. The bucket
machinery is now able to swap out a bucket (close its fastbuf) and
then transparently swap it in as needed.
Martin Mares [Fri, 2 Feb 2007 18:09:59 +0000 (19:09 +0100)]
More bits of the sorter.
In-memory presorting (the general case, but without unification) seems to
work. However, I am not happy with the bucket mechanics yet, it's too
complicated.
Martin Mares [Fri, 2 Feb 2007 12:54:46 +0000 (13:54 +0100)]
big_alloc() now takes a 64-bit argument.
We would like to allocate buffers larger than 4 GB on 64-bit machines.
I've purposefully used u64 instead of size_t, because configuration
variables for buffer sizes will be usually u64's and big_alloc() is
the proper place to report buffer size errors.
Martin Mares [Fri, 2 Feb 2007 11:08:56 +0000 (12:08 +0100)]
The great type cleanup: Use C99 types where applicable.
LibUCW types (u16 and friends) are now based on the C99 <stdint.h> types.
Old-style sbyte, word, sword and addr_int_t are gone (I've removed all refs
to them in the previous commits).
<stddef.h> is now included automatically, bringing types like size_t and
also NULL.
The operations on pointers (GET_P, PUT_P etc.) have been removed from
lib/config.h as nobody uses them and bgetp() is just a duplicate of bgeta()
from lib/fastbuf.h. BYTES_PER_P is gone as well.
#define _GNU_SOURCE had to be moved to lib/config.h, where it belongs anyway.
Martin Mares [Wed, 31 Jan 2007 21:45:41 +0000 (22:45 +0100)]
Several bits of the new sorter.
The basic infrastructure seems to fit well. The 2-way merge module is
adapted from the old sorter and it's definitely worth some more optimization.
The sort-test really works :-)
Pavel Charvat [Mon, 18 Dec 2006 12:02:41 +0000 (13:02 +0100)]
Merged mainline with parallel shep-reap:
- bucket file iface supports more than one opened DB
- shep-reap can preload buckets in parallel (useful especially on RAID systems)
Martin Mares [Mon, 18 Dec 2006 11:29:30 +0000 (12:29 +0100)]
Use gettid() on Linux.
This is a hack for two reasons:
o Current glibc releases (up to 2.5) have no wrapper for gettid(),
so we have to use _syscall0 to access it.
o There is no guarantee that pthreads will correspond to kernel
threads. However, in the two existing implementations of pthreads
on Linux, they correspond either to kernel processes or kernel
threads, so assumptions that the (pid, tid) pair is unique are
correct.
On the other hand, using real thread ID's has many advantages,
so I am keeping this patch at least in the dev-sorter branch to see
if it's safe or not.
Martin Mares [Mon, 11 Dec 2006 22:45:56 +0000 (23:45 +0100)]
Fixed thread safety of signal handling.
Signals delivery is generally not deterministic in multi-threaded programs,
but SIGSEGV is always sent by the kernel to the thread which caused the fault.
Replaced the global signal handler array by a pointer in the per-thread
context and replaced saving of original handles (which was wrong even in
some non-threaded cases) by locked use counters.
Also moved logging of the SIGSEGV error message out of the signal handler
to be sure there are no clashes.
When libucw is compiled without threading, everything is nearly as simple
as before, because the thread context struct is static and all locks
disappear.
Martin Mares [Mon, 11 Dec 2006 22:43:37 +0000 (23:43 +0100)]
Added threading infrastructure to libucw and CONFIG_UCW_THREADS.
Finally decided that we need threading support in too many libucw modules,
so the only sane thing is to link all programs with -lpthread. On the
other hand, I would like to keep the possibility of building non-threaded libucw,
so I've introduced a new config switch, which is always set for Sherlock.
Martin Mares [Sat, 9 Dec 2006 22:24:44 +0000 (23:24 +0100)]
Fixed thread safety of signal handling.
Signals delivery is generally not deterministic in multi-threaded programs,
but SIGSEGV is always sent by the kernel to the thread which caused the fault.
Replaced the global signal handler array by a pointer in the per-thread
context and replaced saving of original handles (which was wrong even in
some non-threaded cases) by locked use counters.
Also moved logging of the SIGSEGV error message out of the signal handler
to be sure there are no clashes.
When libucw is compiled without threading, everything is nearly as simple
as before, because the thread context struct is static and all locks
disappear.
projects / libucw.git / log