]> mj.ucw.cz Git - libucw.git/log
libucw.git
17 years agoCPU_PAGE_SIZE ported from mainline.
Martin Mares [Fri, 12 Jan 2007 11:20:55 +0000 (12:20 +0100)]
CPU_PAGE_SIZE ported from mainline.

17 years agoDetermine system page size in the configure script (introduced CPU_PAGE_SIZE).
Martin Mares [Fri, 12 Jan 2007 11:17:50 +0000 (12:17 +0100)]
Determine system page size in the configure script (introduced CPU_PAGE_SIZE).

This is a work around for broken libc on unstable Debian, but it will
probably be useful elsewhere, too.

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git
Martin Mares [Tue, 2 Jan 2007 14:50:17 +0000 (15:50 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git

(bigalloc got merged to mainline)

17 years agoAdded missing lib/bigalloc.c from dev-async-shep.
Martin Mares [Tue, 2 Jan 2007 14:23:38 +0000 (15:23 +0100)]
Added missing lib/bigalloc.c from dev-async-shep.

17 years agoPartial merge with dev-async-shep:
Pavel Charvat [Tue, 2 Jan 2007 14:12:35 +0000 (15:12 +0100)]
Partial merge with dev-async-shep:
- added routines for allocation of large alligned buffers
- improved random-access utility

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git
Martin Mares [Mon, 18 Dec 2006 13:29:40 +0000 (14:29 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git

(Incorporated all changes from dev-async-shep, v3.10 and v3.9 through
mainline. All merging went smoothly except for lib/threads.c.)

17 years agoadded missing type for a local variable
Pavel Charvat [Mon, 18 Dec 2006 12:23:14 +0000 (13:23 +0100)]
added missing type for a local variable

17 years agoFixed a typo preventing compilation in non-threaded mode.
Martin Mares [Mon, 18 Dec 2006 12:22:00 +0000 (13:22 +0100)]
Fixed a typo preventing compilation in non-threaded mode.

17 years agoRemove relics of pre-CONFIG_UCW_THREADS times.
Martin Mares [Mon, 18 Dec 2006 12:15:28 +0000 (13:15 +0100)]
Remove relics of pre-CONFIG_UCW_THREADS times.

17 years agoMerge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git
Pavel Charvat [Mon, 18 Dec 2006 12:02:50 +0000 (13:02 +0100)]
Merge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git

17 years agoMerged mainline with parallel shep-reap:
Pavel Charvat [Mon, 18 Dec 2006 12:02:41 +0000 (13:02 +0100)]
Merged mainline with parallel shep-reap:
- bucket file iface supports more than one opened DB
- shep-reap can preload buckets in parallel (useful especially on RAID systems)

17 years agoUse thread ID for temporary file creation.
Martin Mares [Mon, 18 Dec 2006 11:03:14 +0000 (12:03 +0100)]
Use thread ID for temporary file creation.

17 years agoAdded thread_id to ucwlib_context.
Martin Mares [Mon, 18 Dec 2006 10:59:35 +0000 (11:59 +0100)]
Added thread_id to ucwlib_context.

17 years agoFree thread context structure on thread termination.
Martin Mares [Mon, 18 Dec 2006 10:52:33 +0000 (11:52 +0100)]
Free thread context structure on thread termination.

17 years agoUse gettid() on Linux.
Martin Mares [Mon, 18 Dec 2006 11:29:30 +0000 (12:29 +0100)]
Use gettid() on Linux.

This is a hack for two reasons:

  o  Current glibc releases (up to 2.5) have no wrapper for gettid(),
     so we have to use _syscall0 to access it.
  o  There is no guarantee that pthreads will correspond to kernel
     threads. However, in the two existing implementations of pthreads
     on Linux, they correspond either to kernel processes or kernel
     threads, so assumptions that the (pid, tid) pair is unique are
     correct.

On the other hand, using real thread ID's has many advantages,
so I am keeping this patch at least in the dev-sorter branch to see
if it's safe or not.

17 years agoUse thread ID for temporary file creation.
Martin Mares [Mon, 18 Dec 2006 11:03:14 +0000 (12:03 +0100)]
Use thread ID for temporary file creation.

17 years agoAdded thread_id to ucwlib_context.
Martin Mares [Mon, 18 Dec 2006 10:59:35 +0000 (11:59 +0100)]
Added thread_id to ucwlib_context.

17 years agoFree thread context structure on thread termination.
Martin Mares [Mon, 18 Dec 2006 10:52:33 +0000 (11:52 +0100)]
Free thread context structure on thread termination.

17 years agoFixed bug in signal handling.
Martin Mares [Fri, 15 Dec 2006 15:02:19 +0000 (16:02 +0100)]
Fixed bug in signal handling.

(The signal handler table needn't be defined in the current thread
at this point.)

17 years agoFixed bug in signal handling.
Martin Mares [Fri, 15 Dec 2006 15:01:36 +0000 (16:01 +0100)]
Fixed bug in signal handling.

17 years agoRemoved the `returned' hook.
Martin Mares [Fri, 15 Dec 2006 13:27:44 +0000 (14:27 +0100)]
Removed the `returned' hook.

17 years agoImplemented priority queues.
Martin Mares [Fri, 15 Dec 2006 13:27:21 +0000 (14:27 +0100)]
Implemented priority queues.

17 years agoASIO requests are not prioritized.
Martin Mares [Fri, 15 Dec 2006 13:25:39 +0000 (14:25 +0100)]
ASIO requests are not prioritized.

17 years agoImplemented priority queues.
Martin Mares [Fri, 15 Dec 2006 13:24:08 +0000 (14:24 +0100)]
Implemented priority queues.

17 years agoKilled struct work.returned, it was a silly interface.
Martin Mares [Tue, 12 Dec 2006 20:45:56 +0000 (21:45 +0100)]
Killed struct work.returned, it was a silly interface.

17 years agoThe asio.test should be performed only in threaded mode.
Martin Mares [Tue, 12 Dec 2006 20:45:33 +0000 (21:45 +0100)]
The asio.test should be performed only in threaded mode.

17 years agoimage-sim-test can display also base224-encoded signatures
Pavel Charvat [Tue, 12 Dec 2006 13:25:07 +0000 (14:25 +0100)]
image-sim-test can display also base224-encoded signatures

17 years agofixed long option for computation of base64 signatures
Pavel Charvat [Tue, 12 Dec 2006 10:23:56 +0000 (11:23 +0100)]
fixed long option for computation of base64 signatures

17 years agoimage-sim-test can display base64-encoded signature
Pavel Charvat [Tue, 12 Dec 2006 10:02:26 +0000 (11:02 +0100)]
image-sim-test can display base64-encoded signature

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git
Martin Mares [Mon, 11 Dec 2006 22:48:31 +0000 (23:48 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git

(After the threading infrastructure has been merged to main.)

17 years agoUpdated various programs and Makefiles to use the threading infrastructure.
Martin Mares [Mon, 11 Dec 2006 22:46:52 +0000 (23:46 +0100)]
Updated various programs and Makefiles to use the threading infrastructure.

17 years agoUpdated libucw threading notes.
Martin Mares [Mon, 11 Dec 2006 22:46:28 +0000 (23:46 +0100)]
Updated libucw threading notes.

17 years agoLogging functions are thread-safe.
Martin Mares [Mon, 11 Dec 2006 22:46:18 +0000 (23:46 +0100)]
Logging functions are thread-safe.

17 years agoFixed thread safety of signal handling.
Martin Mares [Mon, 11 Dec 2006 22:45:56 +0000 (23:45 +0100)]
Fixed thread safety of signal handling.

Signals delivery is generally not deterministic in multi-threaded programs,
but SIGSEGV is always sent by the kernel to the thread which caused the fault.

Replaced the global signal handler array by a pointer in the per-thread
context and replaced saving of original handles (which was wrong even in
some non-threaded cases) by locked use counters.

Also moved logging of the SIGSEGV error message out of the signal handler
to be sure there are no clashes.

When libucw is compiled without threading, everything is nearly as simple
as before, because the thread context struct is static and all locks
disappear.

17 years agoExport temp_file_name().
Martin Mares [Mon, 11 Dec 2006 22:44:55 +0000 (23:44 +0100)]
Export temp_file_name().

17 years agoCreation of temporary files is now thread-safe (ugh).
Martin Mares [Mon, 11 Dec 2006 22:44:43 +0000 (23:44 +0100)]
Creation of temporary files is now thread-safe (ugh).

17 years agoAdded threading infrastructure to libucw and CONFIG_UCW_THREADS.
Martin Mares [Mon, 11 Dec 2006 22:43:37 +0000 (23:43 +0100)]
Added threading infrastructure to libucw and CONFIG_UCW_THREADS.

Finally decided that we need threading support in too many libucw modules,
so the only sane thing is to link all programs with -lpthread. On the
other hand, I would like to keep the possibility of building non-threaded libucw,
so I've introduced a new config switch, which is always set for Sherlock.

17 years agoTrivial cleanups.
Martin Mares [Mon, 11 Dec 2006 22:39:22 +0000 (23:39 +0100)]
Trivial cleanups.

17 years agoAdded a library module for distributing work between threads.
Martin Mares [Mon, 11 Dec 2006 22:38:05 +0000 (23:38 +0100)]
Added a library module for distributing work between threads.

(Ported from the dev-sorter branch.)

17 years agoRemoved an old FIXME.
Martin Mares [Mon, 11 Dec 2006 22:24:28 +0000 (23:24 +0100)]
Removed an old FIXME.

17 years agosize_t != uns
Pavel Charvat [Mon, 11 Dec 2006 20:26:07 +0000 (21:26 +0100)]
size_t != uns

17 years agoFixed thread safety of signal handling.
Martin Mares [Sat, 9 Dec 2006 22:24:44 +0000 (23:24 +0100)]
Fixed thread safety of signal handling.

Signals delivery is generally not deterministic in multi-threaded programs,
but SIGSEGV is always sent by the kernel to the thread which caused the fault.

Replaced the global signal handler array by a pointer in the per-thread
context and replaced saving of original handles (which was wrong even in
some non-threaded cases) by locked use counters.

Also moved logging of the SIGSEGV error message out of the signal handler
to be sure there are no clashes.

When libucw is compiled without threading, everything is nearly as simple
as before, because the thread context struct is static and all locks
disappear.

17 years agoLogging functions are thread-safe.
Martin Mares [Sat, 9 Dec 2006 21:39:36 +0000 (22:39 +0100)]
Logging functions are thread-safe.

17 years agoUse Threads.DefaultStackSize instead of various local stack sizes.
Martin Mares [Sat, 9 Dec 2006 21:16:51 +0000 (22:16 +0100)]
Use Threads.DefaultStackSize instead of various local stack sizes.

17 years agoConfigure default thread stack size globally.
Martin Mares [Sat, 9 Dec 2006 21:10:45 +0000 (22:10 +0100)]
Configure default thread stack size globally.

17 years agoUse the generic thread context to hold per-thread I/O queue.
Martin Mares [Sat, 9 Dec 2006 21:00:38 +0000 (22:00 +0100)]
Use the generic thread context to hold per-thread I/O queue.

17 years agoConverted asio to use the global ucwlib lock.
Martin Mares [Sat, 9 Dec 2006 20:58:13 +0000 (21:58 +0100)]
Converted asio to use the global ucwlib lock.

17 years agoConverted the fb-temp module to use threads.h.
Martin Mares [Sat, 9 Dec 2006 20:57:04 +0000 (21:57 +0100)]
Converted the fb-temp module to use threads.h.

17 years agoAdded a simple threading helper module.
Martin Mares [Sat, 9 Dec 2006 20:55:02 +0000 (21:55 +0100)]
Added a simple threading helper module.

17 years agoRemove extra dependencies on -lpthread.
Martin Mares [Sat, 9 Dec 2006 19:12:57 +0000 (20:12 +0100)]
Remove extra dependencies on -lpthread.

17 years agofb-direct: Implemented temporary files.
Martin Mares [Sat, 9 Dec 2006 19:02:48 +0000 (20:02 +0100)]
fb-direct: Implemented temporary files.

17 years agoDon't forget to build fb-direct.
Martin Mares [Sat, 9 Dec 2006 19:02:33 +0000 (20:02 +0100)]
Don't forget to build fb-direct.

17 years agofb-direct: Name-space cleanup.
Martin Mares [Sat, 9 Dec 2006 19:01:12 +0000 (20:01 +0100)]
fb-direct: Name-space cleanup.

17 years agoCreation of temporary files is now thread-safe (ugh).
Martin Mares [Sat, 9 Dec 2006 19:00:46 +0000 (20:00 +0100)]
Creation of temporary files is now thread-safe (ugh).

17 years agoAdded a direct I/O fastbuf backend.
Martin Mares [Sat, 9 Dec 2006 18:48:39 +0000 (19:48 +0100)]
Added a direct I/O fastbuf backend.

17 years agoAdded CONFIG_UCW_THREADS.
Martin Mares [Sat, 9 Dec 2006 18:47:24 +0000 (19:47 +0100)]
Added CONFIG_UCW_THREADS.

Finally decided that we need threading support in too many libucw modules,
so the only sane thing is to link all programs with -lpthread. On the other
hand, I would like to keep the possibility of building non-threaded libucw,
so I've introduced a new config switch, which is always set for Sherlock.

17 years agoasio: Put a lock around maintenance of use-counts.
Martin Mares [Sat, 9 Dec 2006 18:28:02 +0000 (19:28 +0100)]
asio: Put a lock around maintenance of use-counts.

I want to allow threads creating their own asio queues asynchronously.

17 years agoasio: Add a couple of user-defined fields.
Martin Mares [Sat, 9 Dec 2006 18:27:14 +0000 (19:27 +0100)]
asio: Add a couple of user-defined fields.

17 years agofb-file: A couple of trivial cleanups.
Martin Mares [Sat, 9 Dec 2006 18:26:58 +0000 (19:26 +0100)]
fb-file: A couple of trivial cleanups.

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git
Martin Mares [Fri, 8 Dec 2006 17:09:22 +0000 (18:09 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10
Martin Mares [Fri, 8 Dec 2006 17:09:08 +0000 (18:09 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10

(centrum/cz/{cf/local,config} merged manually)

17 years agoworkqueue: Added non-blocking wait functions.
Martin Mares [Fri, 8 Dec 2006 16:47:16 +0000 (17:47 +0100)]
workqueue: Added non-blocking wait functions.

17 years agoCONFIG_INSTALL_API installs libimages headers
Pavel Charvat [Fri, 8 Dec 2006 16:44:59 +0000 (17:44 +0100)]
CONFIG_INSTALL_API installs libimages headers

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git
Martin Mares [Mon, 4 Dec 2006 12:35:24 +0000 (13:35 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git

17 years agoBenchmark of the new asio-test.
Martin Mares [Fri, 1 Dec 2006 23:00:55 +0000 (00:00 +0100)]
Benchmark of the new asio-test.

17 years agoA more realistic asio-test, which uses memcpy().
Martin Mares [Fri, 1 Dec 2006 23:00:36 +0000 (00:00 +0100)]
A more realistic asio-test, which uses memcpy().

17 years agoMerge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git#dev-sorter
Martin Mares [Fri, 1 Dec 2006 22:25:28 +0000 (23:25 +0100)]
Merge with git+ssh://cvs.ucw.cz/projects/sherlock/GIT/sherlock.git#dev-sorter

17 years agoAdded a couple of ASSERTs more.
Martin Mares [Fri, 1 Dec 2006 22:25:22 +0000 (23:25 +0100)]
Added a couple of ASSERTs more.

17 years agoAdded new benchmarks.
Martin Mares [Fri, 1 Dec 2006 21:52:32 +0000 (22:52 +0100)]
Added new benchmarks.

asio-test turned out to be significantly faster when writing and
slightly faster when copying or reading. Good.

17 years agoAdapted the file-test to use asio.
Martin Mares [Fri, 1 Dec 2006 21:04:38 +0000 (22:04 +0100)]
Adapted the file-test to use asio.

17 years agoSimplified the asio module.
Martin Mares [Fri, 1 Dec 2006 20:36:22 +0000 (21:36 +0100)]
Simplified the asio module.

17 years agoMerge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10
Pavel Charvat [Mon, 27 Nov 2006 23:50:00 +0000 (00:50 +0100)]
Merge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10

17 years agofixed reading of big jpegs (serious today's bug)
Pavel Charvat [Mon, 27 Nov 2006 23:49:09 +0000 (00:49 +0100)]
fixed reading of big jpegs (serious today's bug)

17 years agoMerge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10
Pavel Charvat [Mon, 27 Nov 2006 12:57:32 +0000 (13:57 +0100)]
Merge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10

17 years agofixed deinterlacing of very small GIFs
Pavel Charvat [Mon, 27 Nov 2006 12:41:33 +0000 (13:41 +0100)]
fixed deinterlacing of very small GIFs

17 years agotypo
Pavel Charvat [Mon, 27 Nov 2006 12:11:35 +0000 (13:11 +0100)]
typo

17 years agoMerge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10
Pavel Charvat [Mon, 27 Nov 2006 12:08:59 +0000 (13:08 +0100)]
Merge with git+ssh://git.ucw.cz/projects/sherlock/GIT/sherlock.git#v3.10

17 years agoaccording to libjpeg documentation fill_input_buffer should never return zero
Pavel Charvat [Mon, 27 Nov 2006 12:06:00 +0000 (13:06 +0100)]
according to libjpeg documentation fill_input_buffer should never return zero

17 years agoImplemented an asynchronous I/O library module.
Martin Mares [Sat, 25 Nov 2006 22:35:55 +0000 (23:35 +0100)]
Implemented an asynchronous I/O library module.

It's built at the top of the work queue module.

17 years agoignore libjpeg warnings
Pavel Charvat [Sat, 25 Nov 2006 21:14:31 +0000 (22:14 +0100)]
ignore libjpeg warnings

17 years ago64bit bug in image signatures
Pavel Charvat [Sat, 25 Nov 2006 21:11:15 +0000 (22:11 +0100)]
64bit bug in image signatures

17 years agoAdded a library module for distributing work between threads.
Martin Mares [Sat, 25 Nov 2006 21:10:14 +0000 (22:10 +0100)]
Added a library module for distributing work between threads.

However, I am not sure that libucw is the right place for this, since
it would require all programs to be linked with libpthread as well.

17 years agofixed segfaulting libjpeg warnings when decoding partially wrong images...
Pavel Charvat [Fri, 24 Nov 2006 23:31:25 +0000 (00:31 +0100)]
fixed segfaulting libjpeg warnings when decoding partially wrong images...
now we reject such images, but it can be easily changed in ifdef

17 years agoThe first idea of an async I/O interface.
Martin Mares [Fri, 24 Nov 2006 20:32:17 +0000 (21:32 +0100)]
The first idea of an async I/O interface.

17 years agoMore benchmark results.
Martin Mares [Fri, 24 Nov 2006 18:52:58 +0000 (19:52 +0100)]
More benchmark results.

17 years agoMuch better parallel copy benchmark.
Martin Mares [Fri, 24 Nov 2006 18:07:03 +0000 (19:07 +0100)]
Much better parallel copy benchmark.

17 years agoUpdated the sorter usage map.
Martin Mares [Fri, 24 Nov 2006 15:30:01 +0000 (16:30 +0100)]
Updated the sorter usage map.

17 years agoImproved the file benchmark slightly.
Martin Mares [Fri, 24 Nov 2006 00:04:31 +0000 (01:04 +0100)]
Improved the file benchmark slightly.

17 years agoAdded a simple experiment with parallel reads/writes.
Martin Mares [Thu, 23 Nov 2006 23:33:24 +0000 (00:33 +0100)]
Added a simple experiment with parallel reads/writes.

17 years agoUse big_alloc().
Martin Mares [Thu, 23 Nov 2006 23:22:11 +0000 (00:22 +0100)]
Use big_alloc().

17 years agoAdded a couple of notes.
Martin Mares [Thu, 23 Nov 2006 23:18:08 +0000 (00:18 +0100)]
Added a couple of notes.

17 years agoAdded big_alloc() for allocating large, page-aligned buffers.
Martin Mares [Thu, 23 Nov 2006 23:17:17 +0000 (00:17 +0100)]
Added big_alloc() for allocating large, page-aligned buffers.

17 years agoMoved tests related to the sorter to debug/sorter/.
Martin Mares [Thu, 23 Nov 2006 18:06:26 +0000 (19:06 +0100)]
Moved tests related to the sorter to debug/sorter/.

17 years agopartial YCCK colorspace support
Pavel Charvat [Sat, 11 Nov 2006 20:11:11 +0000 (21:11 +0100)]
partial YCCK colorspace support

17 years agofixed Bug 3188 - libimages supports shared libraries;
Pavel Charvat [Mon, 6 Nov 2006 11:29:11 +0000 (12:29 +0100)]
fixed Bug 3188 - libimages supports shared libraries;
it depends on libsh only with CONFIG_SHERLOCK

17 years agohacked libimages tests dependencies
Pavel Charvat [Mon, 6 Nov 2006 10:34:38 +0000 (11:34 +0100)]
hacked libimages tests dependencies

17 years agofixed color.test compilation
Pavel Charvat [Mon, 6 Nov 2006 09:52:49 +0000 (10:52 +0100)]
fixed color.test compilation

17 years agoMoved the update hook script to a separate repository.
Martin Mares [Fri, 3 Nov 2006 19:41:28 +0000 (20:41 +0100)]
Moved the update hook script to a separate repository.

17 years agoadded support for some color conversions; we should be able to load cmyk
Pavel Charvat [Thu, 2 Nov 2006 23:06:50 +0000 (00:06 +0100)]
added support for some color conversions; we should be able to load cmyk
jpegs now...

17 years agoimage-tool uses RGB automatically when converting non-RGB jpegs
Pavel Charvat [Wed, 1 Nov 2006 13:52:57 +0000 (14:52 +0100)]
image-tool uses RGB automatically when converting non-RGB jpegs
to gif/png