Soft heaps: Example.

[saga.git] / opt.tex

\ifx\endpart\undefined
\input macros.tex
\fi

\chapter{Approaching Optimality}

\section{Soft heaps}

Recently, Chazelle \cite{chazelle:ackermann} and Pettie \cite{pettie:ackermann}
have presented algorithms for the MST with worst-case time complexity
$\O(m\timesalpha(m,n))$. We will devote this chapter to their results
and especially to another algorithm by Pettie and Ramachandran \cite{pettie:optimal},
which is provably optimal.

At the very heart of all these algorithms lies the \df{soft heap} discovered by
Chazelle \cite{chazelle:softheap}. It is a~meldable priority queue, roughly
similar to the Vuillemin's binomial heaps \cite{vuillemin:binheap} or Fredman's
and Tarjan's Fibonacci heaps \cite{ft:fibonacci}. The soft heaps run faster at
the expense of \df{corrupting} a~fraction of the inserted elements by raising
their values (the values are however never decreased). This allows for
an~trade-off between accuracy and speed controlled by a~parameter~$\varepsilon$.
The heap operations take $\O(\log(1/\varepsilon))$ amortized time and at every
moment at most~$\varepsilon n$ elements of the~$n$ elements inserted can be
corrupted.

\defnn{Soft heap operations}%
The soft heap contains distinct elements from a~totally ordered universe and it
supports the following operations:
\itemize\ibull
\:$\<Create>(\varepsilon)$ --- create an~empty soft heap with the given accuracy parameter,
\:$\<Insert>(H,x)$ --- insert a~new element~$x$ to the heap~$H$,
\:$\<Meld>(P,Q)$ --- merge two heaps into one, more precisely insert all elements of a~heap~$Q$
  to the heap~$P$, destroying~$Q$ in the process (both heaps must have the same~$\varepsilon$),
\:$\<DeleteMin>(H)$ --- delete the minimum element of the heap~$H$ and return its value
  (optionally signalling that the value has been corrupted).
\endlist

\examplen{Linear-time selection}
We can use soft heaps to select the median (or generally the $k$-th smallest element)
of a~sequence. We insert all $n$~elements to a~soft heap with error rate $\varepsilon=1/3$.
Then we delete the minimum about $n/3$ times and we remember the current (possibly corrupted)
value~$x$ of the last element deleted. This~$x$ is greater or equal than the current
values of the previously deleted elements and thus also than their correct values.
On the other hand, the current values of the $2n/3$ elements remaining in the heap
are greater than~$x$ and at most $n/3$ of them are corrupted. Therefore at least $n/3$
elements are greater than~$x$ and at least $n/3$ ones are smaller. Depending on the value
of~$k$, we can always throw away one of these parts and recurse on the rest (similarly to
the Hoare's Quickselect algorithm \cite{hoare:qselect}). The total time complexity will
be $\O(n+(2/3)\cdot n+(2/3)^2\cdot n+\ldots) = \O(n)$. We have obtained a~nice alternative to the standard
linear-time selection algorithm by Blum \cite{blum:selection}. The same trick can be used
to select a~good pivot in Quicksort \cite{hoare:qsort}, leading to time complexity $\O(n\log n)$
in the worst case.

\defnn{Soft queues}
The soft heap is built from \df{soft queues} (we will omit the adjective soft
in the rest of this section). Each queue has a~shape of a~binary tree.\foot{%
Actually, Chazelle defines the queues as binomial trees, but he transforms them in ways that are
somewhat counter-intuitive, albeit well-defined. We prefer describing the queues as binary
trees with a~special distribution of values. In fact, the original C~code in the Chazelle's
paper uses this representation internally.}
Each vertex~$v$ of the tree remembers a~doubly-linked list $\<list>(v)$ of values. The
item list in every left son will be used only temporarily and it will be kept
empty between operations. Only right sons and the root have their lists
permanently occupied. The former vertices will be called \df{white}, the latter
\df{black.} (See the picture.)

The first value in the list is called the \df{controlling key} of the vertex,
denoted by $\<ckey>(v)$. If the list is empty, we keep the most recently used
value or we set $\<ckey>(v)=+\infty$. The \<ckey>'s obey the standard \df{heap order}
--- a~\<ckey> of a~parent is always less than the \<ckey>'s of its sons.

Each vertex is also assigned its \df{rank,} which is a~non-negative integer.
The rank of leaves is always zero, the rank of every internal vertex must be strictly
greater than the ranks of its sons. We define the rank of a~queue to be equal to the
rank of its root vertex and similarly for its \<ckey>.

A~queue is called \df{complete} if every two vertices joined by an~edge have rank
difference exactly one. In other words, it is a~complete binary tree and the
ranks correspond to vertex heights.

\figure{softheap1.eps}{\epsfxsize}{\multicap{A~complete and a~partial soft queue tree\\
(black vertices contain items, numbers indicate ranks)}}

\obs
A~complete queue of rank~$k$ contains exactly~$2^{k+1}-1$ vertices, $2^k$~of which are
black (by induction). Any other queue can be trivially embedded to a~complete queue of the same
rank, which we will call the \df{master tree} of the queue. This embedding preserves vertex
ranks, colors and the ancestor relation.

The queues have a~nice recursive structure. We can construct a~queue of rank~$k$ by \df{joining}
two queues of rank~$k-1$ under a~new root. The root will inherit the item list of one
of the original roots and also its \<ckey>. To preserve the heap order, we will choose
the one whose \<ckey> is smaller.

Sometimes, we will need to split a~queue to smaller queues. We will call this operation
\df{dismantling} the queue and it will happen only in cases when the item list in the root
is empty. It suffices to remove the leftmost (all white) path going from the root. From
a~queue of rank~$k$, we get queues of ranks $0,1,\ldots,k-1$, some of which may be missing
if the original queue was not complete.

We will now combine the soft queues to the soft heap.

\figure{softheap2.eps}{\epsfxsize}{Joining and dismantling of soft queues}

\defn A~\df{soft heap} consists of:
\itemize\ibull
\:a~doubly linked list of soft queues of distinct ranks (in increasing order of ranks),
  we will call the first queue the \df{head}, the last queue will be its \df{tail};
\:\df{suffix minima:} each queue contains a~pointer to the queue with minimum \<ckey>
of those following it in the list;
\:and a~global parameter~$r\in{\bb N}$, to be set depending on~$\varepsilon$.
\endlist

\>We will define the \df{rank} of a~heap as the highest of the ranks of its queues
(that is, the rank of the heap's tail).

The heap always keeps the \df{Rank invariant:} When a~root of any tree has rank~$k$,
its leftmost path has length at least~$k/2$.

\para
\em{Melding} of two soft heaps involves merging of their lists of queues. We disassemble
the heap with the smaller maximum rank and we insert its queues to the other heap.
If two queues of the same rank~$k$ appear in both lists, we \em{join} them to
a~single queue of rank~$k+1$ as already described and we propagate the new queue as
a~\df{carry} to the next iteration, similarly to addition of binary numbers.
Finally we have to update the suffix minima by walking the new list backwards
from the last inserted item.

\em{Creation} of a~new soft heap is trivial, \em{insertion} is handled by creating
a~single-element heap and melding it to the destination heap.

\algn{Creating a~new soft heap}
\algo
\algin A~parameter~$\varepsilon$.
\:Allocate memory for a~new heap structure~$H$.
\:Initialize the list of queues in~$H$ as an~empty list.
\:Set the parameter~$r$ to~$2\lceil\log(1/\varepsilon)\rceil+2$ (to be justified later).
\algout A~newly minted soft heap~$H$.
\endalgo

\algn{Melding of two soft heaps}
\algo
\algin Two soft heaps~$P$ and~$Q$.
\:If the heap~$P$ has smaller rank than the heap~$Q$, exchange their item lists.
\:$p\=\<head>(P)$.
\brk\cmt{Whenever we run into an~end of a~list in this procedure, we assume that it contains
an~empty queue of infinite rank.}
\:While $Q$ still has some queues:
\::$q\=\<head>(Q)$.
\::If $\<rank>(p) < \<rank>(q)$, then $p\=$ the successor of~$p$,
\::else if $\<rank>(p) > \<rank>(q)$, remove~$q$ from~$Q$ and insert it before~$p$,
\::otherwise (the ranks are equal, we need to propagate the carry):
\:::$\<carry>=p$.
\:::$p\=$ the successor of~$p$.
\:::While $\<rank>(q)=\<rank>(\<carry>)$:
\::::Remove~$q$ from~$Q$.
\::::$\<carry>\=\<join>(q, \<carry>)$.
\::::$q\=\<head>(Q)$.
\:::Insert~\<carry> before~$q$.
\:Update the suffix minima: Walk with~$p$ backwards to the head of~$P$:
\::$p'\=$ successor of~$p$.
\::If $\<ckey>(p) < \<ckey>(p')$, set $\<suffix\_min>(p)\=p$.
\::Otherwise set $\<suffix\_min>(p)\=\<suffix\_min>(p')$.
\:Destroy the heap~$Q$.
\algout The merged heap~$P$.
\endalgo

\algn{Insertion of an~element to a~soft heap}
\algo
\algin A~heap~$H$ and a~new element~$x$.
\:Create a~new heap~$H'$ with the same parameters as~$H$. Let~$H'$ contain a~sole queue of rank~0,
  whose only vertex has the element~$x$ in its item list.
\:$\<Meld>(H,H')$.
\algout An~updated heap~$H$.
\endalgo

\para
So far, the mechanics of the soft heaps was almost identical to the binomial heaps
and the reader could rightfully yawn. The things are going to get interesting
now as we approach the parts where corruption of items takes place.

If all item lists contain at most one item equal to the \<ckey> of the particular
vertex, no information is lost and the heap order guarantees that the minimum item
of every queue stays in its root. We can however allow longer lists and let the items
stored in a~single list travel together between the vertices of the tree, still
represented by a~common \<ckey>. This data-structural analogue of car pooling will
allow the items to travel at a~faster rate, but as only a~single item can be equal
to the \<ckey>, all other items will be inevitably corrupted.

We of course have to be careful about the size of the lists, as we must avoid corrupting
too many items. We will control the growth according to the vertex ranks. Vertices
with rank at most~$r$ will always contain just a~single item. Above this level,
the higher is the rank, the longer list will we allow.

\para
\em{Deleting of minimum} will follow this principle. The minimum is easy to locate
--- we follow the \<suffix\_min> of the head of the heap to the queue with the minimum
\<ckey> and we look inside the item list of the root of this queue. We remove the \em{last}
item from the list (we do not want the \<ckey> to change) and return it as the minimum.
(It is not necessarily the real minimum of all items, but always the minimum of those
uncorrupted.)

If the list becomes empty, we \em{refill} it with items from the lower levels of the
same tree. This can be best described recursively: We ask the left son to refill itself
(remember that the left son is always white, so there are no items to lose). If the new
\<ckey> of the left son is smaller than of the right son, we immediately move the left
son's list to its parent. Otherwise, we exchange the sons and move the list from the
new left son to the parent. This way, we obey the heap order and the same time we keep
the white left son free of items.

Ocassionally, we repeat this process once again and we concatenate the resulting lists
(we append the latter list to the former, using the smaller of the two \<ckey>'s). This
makes the lists grow longer and we want to do that roughly on every other level of the
tree. The exact condition will be that either the rank of the current vertex is odd,
or the difference in ranks between this vertex and its right son is at least two.

If refilling of the left son fails because there are no more items in that subtree
(we report this by setting its \<ckey> to $+\infty$), the current vertex is no longer
needed, as the items would just travel through it unmodified. We therefore want to
remove it. Instead of deleting it directly, we will rather make it point to its former
grandsons and we will remove the (now orhpaned) original son. This helps us to ensure
that both sons always have the same rank, which will be useful for the analysis.

When all refilling is done, we update the suffix minima by walking from the current
queue to the head of the heap exactly as we did in the \<Meld> procedure.

Our only remaining worry is that the Rank invariant can be broken after the
refilling. When the leftmost path of the tree becomes too short, we just
\em{dismantle} the tree as already described and we meld the new trees back to
the heap. This is easier to handle when the item list at the root vertex is
empty. We will therefore move this check before the refilling of the root list.
It will turn out that we have enough time to always walk the leftmost path
completely, so no explicit counters are needed.

Let us translate these ideas to real (pseudo)code:

\algn{Deleting the minimum item of a~soft heap}
\algo
\algin A~soft heap~$H$.
\:Use \<suffix\_min> of the head queue of~$H$ to locate the queue~$q$ with the smallest \<ckey>.
\:Remove the last element~$x$ of the item list in the root of~$q$.
\:If the item list is empty:
\::Count the vertices on the leftmost path of~$q$.
\::If there are less than $\<rank>(q)$ of them:
\:::Remove~$q$ from the list of queues.
\:::Recalculate the suffix minima as in the \<Meld> procedure.
\:::Dismantle~$q$ and create a~heap~$H'$ holding the resulting trees.
\:::Meld them back: $\<Meld>(H,H')$.
\::Otherwise:
\:::Call \<Refill> on the root of~$q$.
\:::If $\<ckey>(q)=+\infty$ (no items left), remove the tree~$q$.
\:::Recalculate the suffix minima.
\algout The deleted minimum item~$x$ (possibly corrupted).
\endalgo

\algn{Refilling the item list of a~vertex}
\algo
\algin A~soft queue and its vertex~$v$ with an~empty item list.
\:Handle trivial cases: If~$v$ has no children or both have $\<ckey>=+\infty$,
  set $\<ckey>(v)$ to~$+\infty$ and return.
\:Let \<left> and~\<right> denote the sons of~$v$.
\:Recurse: call $\<Refill>(\<left>)$.
\:If $\<ckey>(\<left>) > \<ckey>(\<right>)$, swap the sons.
\:Move the item list from \<left> to~$v$ (implying $\<ckey>(v)=\<ckey>(\<left>)$).
\:If $\<rank>(v) > r$ and either $\<rank>(v)$ is odd or $\<rank>(v) > \<rank>(\<right>)+1$, recurse once more:
\::Repeat steps 3--4.
\::Append the item list from \<left> to the item list at~$v$.
\:Clean up. If $\<ckey>(\<right>) = +\infty$:
\::If $\<ckey>(\<left>) = +\infty$, unlink and discard both sons.
\::Otherwise relink the sons of \<left> to~$v$ and discard \<left>.
\algout A~modified soft queue.
\endalgo

\paran{Analysis of accuracy}
The description of the operations is complete, let us analyse their behavior
and verify that we have delivered what we promised --- first the accuracy of
the structure, then the time complexity of operations. In the whole analysis,
we will denote the total number of elements inserted in the history of the
structure by~$n$. We will also assume that the threshold~$r$ is even.

We start by bounding the size of the item lists.

\lemma
For every vertex~$v$ of a~soft queue, the size $\ell(v)$ of its item list
satisfies:
$$\ell(v) \le \max(1, 2^{\lceil \<rank>(v)/2 \rceil - r/2}).$$

\proof
Initially, all item lists contain at most one item, so the ineqality trivially
holds. Let us continue by induction. Melds can affect it only in the favorable
direction (they sometimes move an~item list to a~vertex of a~higher rank)
and so do deletes (they only remove items from lists). The only potentially
dangerous place is the \<Refill> procedure.

Refilling sometimes just moves items upwards, which is safe, and sometimes it
joins two lists into one, which generally is not. When $\<rank>(v) \le r$,
no joining takes place and~$\ell(v)$ is still~1. Otherwise we join when either
$\<rank>(v)$ is odd or $\<rank>(w) < \<rank>(v)-1$ for any son~$w$ of~$v$ (remember
that both sons have the same rank). In both cases, $\lceil\<rank>(w)/2\rceil \le
\lceil\<rank>(v)/2\rceil - 1$. By the induction hypothesis, the size of each
of the two joined lists is at most $2^{\max(1,\lceil\<rank>(v)/2\rceil - 1 - r/2)}$,
so the result has at most $2^{\lceil\<rank>(v)/2\rceil - r/2}$ items. (The maximum
has disappeared since $\<rank>(v)>r$ and therefore the desired bound is at least~2.)
\qed

We will now sum the sizes of the lists over all vertices containing corrupted items.

\lemma\id{shcorrlemma}%
At any given time, the heap contains at most~$n/2^{r-2}$ corrupted items.

\proof
We first prove an~auxiliary claim: The master trees of all queues contain at most~$n$
black vertices. This follows by induction: If no deletions have taken place,
there are exactly~$n$ of them, because insertion adds one black vertex and
melding preserves their number. A~deletion affects the master trees only when
dismantling takes place and then it only removes a~black vertex.

An~obvious upper bound on the number of corrupted items is the total size of item
lists in all vertices of rank greater than~$r$. We already know from the previous
lemma that the list sizes are limited by a~function of the ranks. A~complete tree
is obviously the worst case, so we will prove that this lemma holds for the master
tree of every queue in the heap. The actual trees can be much sparser, but the
above claim guarantees that the total size of the master trees is bounded by the
number of insertions properly.

So let us consider a~complete tree of~rank~$k$. It has exactly $2^{k-i}$ vertices
of rank~$i$ and each such vertex contains a~list of at most~$2^{\lceil i/2\rceil - r/2}$
items by the previous lemma. Summing over all ranks greater than~$r$, we get that
the total number of corrupted items in this tree is at most:
$$
\sum_{i=r+1}^k 2^{k-i}\cdot 2^{\lceil i/2\rceil - r/2}
= 2^{k-r/2} \cdot \sum_{i=r+1}^k 2^{\lceil i/2\rceil - i}
\le 2^{k-r/2} \cdot \sum_{i=r+1}^k 2^{-i/2}
\le 2^{k-r} \cdot \sum_{i=0}^\infty 2^{-i/2}.
$$
The sum of a~geometric series with quotient $2^{-1/2}$ is less than four, so the
last formula is less than $2^{k-r+2}$. Since the tree contains $n_k=2^k$ black vertices,
this makes less than $n_k/2^{k-2}$ corrupted items as we asserted.
\qed

\paran{Analysis of time complexity}
Now we will analyse the amortized time complexity of the individual operations.
We will show that if we charge $\O(r)$ time against every element inserted, it suffices
to cover the cost of all other operations. We take a~look at the melds first.

\lemma\id{shmeld}%
The amortized cost of a~meld is $\O(1)$, except for melds induced by dismantling
which take $\O(\<rank>(q))$, where $q$~is the queue to be dismantled.

\proof
The real cost of a~meld of heaps $P$ and~$Q$ is the smaller of their ranks plus
the time spent on carry propagation. The latter is easy to dispose of: since
every time there is a~carry, the total number of trees in all heaps decreases
by one, it suffices to charge $\O(1)$ against creation of a~tree. An~insert creates
one tree, dismantling creates at most $\<rank>(q)$ trees, all other operations
alter only the internal structure of trees.

As for the $\O(\min(\<rank>(P),\<rank>(Q)))$ part, let us assume for a~while that
no dismantling ever takes place and consider the \df{meld forest.} It is a~forest
whose leaves correspond to the $n$~single-element heaps constructed by \<Insert>
and each internal vertex represents a~heap arisen from melding its sons. The left
son will be the one with the greater (or equal) rank. We therefore want to bound
the sum of ranks of all right sons.

For every right son, we will distribute the change for its rank~$k$ on all leaves
in its subtree. There are at least $2^k$ such leaves. No leaf ever receives the same
rank twice, because the ranks of right sons on every path from the root of the
tree to a~leaf are strictly decreasing. (This holds because melding two heaps
of the same rank always produces a~heap of higher rank.) Hence at most~$n/2^k$
right sons have rank~$k$ and the total time charged against the leaves is bounded by:
$$
\sum_{k=0}^{\rm max. rank}k\cdot {n\over 2^k} \le n\cdot\sum_{k=0}^\infty {k\over 2^k} = \O(n).
$$

Let us return dismantling to the game. When a~queue is dismantled, melding the parts
back to the heap takes $\O(\<rank>(q))$ time. We can therefore let the dismantling pay for it
and omit such induced melds from the meld forest. As the rank of the heap is never increased
by induced melds, the above calculation is still a~proper upper bound on the cost
of the regular melds.
\qed

To estimate the time spent on deletions, we analyse the refills first.

\lemma
Every call of the \<Refill> procedure spends time $\O(1)$ amortized.

\proof
When \<Refill> is called from the \<DeleteMin> operation, it recurses on a~subtree of the
queue. This subtree can be split to the ``lossless'' lower part (rank~$r$ and below)
and the upper part where list concatenation and thus also corruption takes place. Whenever
we visit the lower part during the recursion, we spend at worst $\O(r)$ time there.
We will prove that the total time spent in the upper parts during the whole life of the
data structure is $\O(n)$. Since each upper vertex can perform at most two calls to the lower
part, the total time spent in the lower parts is $\O(rn)$. All this can be pre-paid by the
inserts.

Let us focus on the upper part. There are three possibilities of what can happen
when we visit a~vertex:

\itemize\ibull

\:We delete it: Every vertex deleted has to have been created at some time in the past.
New vertices are created only during inserts and melds (when joining two trees) and
we have already shown that these operations have constant amortized complexity. So the
same must hold for deletions.

\:We recurse twice and concatenate the lists: The lists are disassembled only when
they reach the root of the tree, otherwise they are only concatenated. We can easily
model the situation by a~binary tree forest similar to the meld forest. There are~$n$
leaves and every internal vertex has outdegree two, so the total number of concatenations
is at most~$n$. Each of them can be performed in constant time as the list is doubly linked.

\:We recurse only once: This occurs only if the rank is even and the gap between the
rank of this vertex and its sons is small. It therefore cannot happen twice in a~row,
so it is clearly dominated by the other possibilities.

\endlist
\>The total cost of all operations on the upper part is therefore $\O(n)$.
\qed

It remains to examine the rest of the \<DeleteMin> operation.

\lemma\id{shdelmin}%
Every \<DeleteMin> takes $\O(1)$ time amortized.

\proof
Aside from refilling, which is $\O(1)$ by the previous lemma, the \<DeleteMin>
takes care of the Rank invariant. This happens by checking the length of the leftmost
path and dismantling the tree if the length is too far from the tree's rank~$k$.
The leftmost path is however always visited by the call to \<Refill>, so we can
account the check on the refilling. The same holds for disassembling. We then have
to pay $\O(k)$ for melding the trees back to the heap, but since there are at most
$k/2$ trees, a~subtree of rank at least $k/2$ must have been deleted. This tree
contained at least $2^{k/2}$ vertices which are now permanently gone from the
data structure, so we can charge the cost of the meld against these vertices.

We must not forget that \<DeleteMin> also has to recalculate the suffix minima.
In the worst case, it requires touching $k$~trees. Because of the Rank invariant,
this is linear in the size of the
leftmost path and therefore it can be also paid for by \<Refill>. (Incidentally,
this was the only place where we needed the invariant.)
\qed

Now we can put the bits together and laurel our effort with the following theorem:

\thmn{Performance of soft heaps, Chazelle \cite{chazelle:softheap}}
A~soft heap with error rate~$\varepsilon$ ($0<\varepsilon\le 1/2$) processes
a~sequence of operations starting with an~empty heap and containing $n$~\<Insert>s
in time $\O(n\log(1/\varepsilon))$. At every moment, the heap contains at most
$\varepsilon n$ corrupted items.

\proof
We set the parameter~$r$ to~$2+2\lceil\log (1/\varepsilon)\rceil$. The rest follows
from the analysis above. By Lemma \ref{shcorrlemma}, there are always at most $n/2^{r-2}
\le \varepsilon n$ corrupted items in the heap. By Lemma \ref{shmeld}--\ref{shdelmin},
the time spent on all operations in the sequence can be paid for by charging $\O(r)$ time
against each \<Insert>, which yields the time bound.
\qed

\FIXME{Remark on optimality.}

\FIXME{Example of use: pivots.}



\endpart