Soft heaps: fixes.

[saga.git] / opt.tex

\ifx\endpart\undefined
\input macros.tex
\fi

\chapter{Approaching Optimality}

\section{Soft heaps}

Recently, Chazelle \cite{chazelle:ackermann} and Pettie \cite{pettie:ackermann}
have presented algorithms for the MST with worst-case time complexity
$\O(m\timesalpha(m,n))$. We will devote this chapter to their results
and especially to another algorithm by Pettie and Ramachandran \cite{pettie:optimal},
which is provably optimal.

At the very heart of all these algorithms lies the \df{soft heap} discovered by
Chazelle \cite{chazelle:softheap}. It is a~meldable priority queue, roughly
similar to the Vuillemin's binomial heaps \cite{vuillemin:binheap} or Fredman's
and Tarjan's Fibonacci heaps \cite{ft:fibonacci}. The soft heaps run faster at
the expense of \df{corrupting} a~fraction of the inserted elements by raising
their values (the values are however never lowered). This allows for
an~trade-off between accuracy and speed controlled by a~parameter~$\varepsilon$.
The heap operations take $\O(\log(1/\varepsilon))$ amortized time and at every
moment at most~$\varepsilon n$ elements of the~$n$ elements inserted can be
corrupted.

\defnn{Soft heap operations}%
The soft heap contains a~set of distinct items from a~totally ordered universe and it
supports the following operations:
\itemize\ibull
\:$\<Create>(\varepsilon)$ --- create an~empty soft heap with the given accuracy parameter~$\varepsilon$,
\:$\<Insert>(H,x)$ --- insert a~new element~$x$ to the heap~$H$,
\:$\<Meld>(P,Q)$ --- merge two heaps into one, more precisely move all elements of a~heap~$Q$
  to the heap~$P$, destroying~$Q$ in the process (both heaps must have the same~$\varepsilon$),
\:$\<DeleteMin>(H)$ --- delete the minimum element of the heap~$H$ and return its value
  (optionally signalling that the value has been corrupted).
\endlist

\examplen{Linear-time selection}
We can use soft heaps to select the median (or generally the $k$-th smallest element)
of a~sequence. We insert all $n$~elements to a~soft heap with error rate $\varepsilon=1/3$.
Then we delete the minimum about $n/3$ times and we remember the current (possibly corrupted)
value~$x$ of the last element deleted. This~$x$ is greater or equal than the current
values of the previously deleted elements and thus also than their correct values.
On the other hand, the current values of the $2n/3$ elements remaining in the heap
are greater than~$x$ and at most $n/3$ of them are corrupted. Therefore at least $n/3$
elements are greater than~$x$ and at least $n/3$ ones are smaller. Depending on the value
of~$k$, we can always throw away one of these parts and recurse on the rest (similarly to
the Hoare's Quickselect algorithm \cite{hoare:qselect}). The total time complexity will
be $\O(n+(2/3)\cdot n+(2/3)^2\cdot n+\ldots) = \O(n)$. We have obtained a~nice alternative to the standard
linear-time selection algorithm by Blum \cite{blum:selection}. The same trick can be used
to select a~good pivot in Quicksort \cite{hoare:qsort}, leading to time complexity $\O(n\log n)$
in the worst case.

\defnn{Soft queues}
The soft heap is built from \df{soft queues} (we will usually omit the adjective soft
in the rest of this section). Each queue has a~shape of a~binary tree.\foot{%
Actually, Chazelle defines the queues as binomial trees, but he transforms them in ways that are
somewhat counter-intuitive, albeit well-defined. We prefer describing the queues as binary
trees with a~special distribution of values. In fact, the original C~code in the Chazelle's
paper \cite{chazelle:softheap} uses this representation internally.}
Each vertex~$v$ of the tree remembers a~doubly-linked list of items. The
item list in every left son will be used only temporarily and it will be kept
empty between operations. Only right sons and the root have their lists
permanently occupied. The left sons will be called \df{white}, the right ones
\df{black.} (See the picture.)

The first value in every list is called the \df{controlling key} of the vertex,
denoted by $\<ckey>(v)$. If the list is empty, we keep the most recently used
value or we set $\<ckey>(v)=+\infty$. The \<ckey> obey the standard \df{heap order}
--- a~\<ckey> of a~parent is always smaller than the \<ckey>'s of its sons.

Each vertex is also assigned its \df{rank,} which is a~non-negative integer.
The ranks of leaves are always zero, the rank of every internal vertex can be
arbitrary, but it must be strictly greater than the ranks of its sons. We
define the rank of the whole queue to be equal to the rank of its root vertex and
similarly for its \<ckey>.

A~queue is called \df{complete} if every two vertices joined by an~edge have rank
difference exactly one. In other words, it is a~complete binary tree and the
ranks correspond to vertex heights.

\figure{softheap1.eps}{\epsfxsize}{\multicap{A~complete and a~partial soft queue tree\\
(black vertices contain items, numbers indicate ranks)}}

\obs
The complete queue of rank~$k$ contains exactly~$2^{k+1}-1$ vertices, $2^k$~of which are
black (by induction). Any other queue can be trivially embedded to the complete queue of the same
rank, which we will call the \df{master tree} of the queue. This embedding preserves vertex
ranks, colors and the ancestor relation.

The queues have a~nice recursive structure. We can construct a~queue of rank~$k$ by \df{joining}
two queues of rank~$k-1$ under a~new root. The root will inherit the item list of one
of the original roots and also its \<ckey>. To preserve the heap order, we will choose
the one whose \<ckey> is smaller.

Sometimes, we will also need to split a~queue to smaller queues. We will call this operation
\df{dismantling} the queue and it will happen only in cases when the item list in the root
is empty. It suffices to remove the leftmost (all white) path going from the root. From
a~queue of rank~$k$, we get queues of ranks $0,1,\ldots,k-1$, some of which may be missing
if the original queue was not complete.

\figure{softheap2.eps}{\epsfxsize}{Joining and dismantling of soft queues}

We will now define the real soft heap and the operations on it.

\defn A~\df{soft heap} consists of:
\itemize\ibull
\:a~doubly linked list of soft queues of distinct ranks (in increasing order of ranks),
  we will call the first queue the \df{head} of the list, the last queue will be its \df{tail};
\:\df{suffix minima:} each queue contains a~pointer to the queue with minimum \<ckey>
of those following it in the list;
\:a~global parameter~$r$: an~even integer to be set depending on~$\varepsilon$.
\endlist

\>We will define the \df{rank} of a~heap as the highest of the ranks of its queues
(that is, the rank of the heap's tail).

The heap always keeps the \df{Rank invariant:} When a~root of any tree has rank~$k$,
its leftmost path contains at least~$k/2$ vertices.

\paran{Operations on soft heaps}

\em{Melding} of two soft heaps involves merging of their lists of queues. We disassemble
the heap of the smaller rank and we insert its queues to the other heap.
If two queues of the same rank~$k$ appear in both lists, we \em{join} them to
a~single queue of rank~$k+1$ as already described and we propagate the new queue as
a~\df{carry} to the next iteration, similarly to addition of binary numbers.
Finally we have to update the suffix minima by walking the new list backwards
from the last inserted item.

\em{Creation} of a~new soft heap is trivial, \em{insertion} is handled by creating
a~single-element heap and melding it to the destination heap.

\algn{Creating a~new soft heap}
\algo
\algin The~parameter~$\varepsilon$ (the accuracy of the heap).
\:Allocate memory for a~new heap structure~$H$.
\:Initialize the list of queues in~$H$ to an~empty list.
\:Set the parameter~$r$ to~$2\lceil\log(1/\varepsilon)\rceil+2$ (to be justified later).
\algout A~newly minted soft heap~$H$.
\endalgo

\algn{Melding of two soft heaps}
\algo
\algin Two soft heaps~$P$ and~$Q$.
\:If $\<rank>(P) < \<rank>(Q)$, exchange the item lists of~$P$ and~$Q$.
\:$p\=\<head>(P)$.
\brk\cmt{Whenever we run into an~end of a~list in this procedure, we assume that
there is an~empty queue of infinite rank there.}
\:While $Q$ still has some queues:
\::$q\=\<head>(Q)$.
\::If $\<rank>(p) < \<rank>(q)$, then $p\=$ the successor of~$p$,
\::else if $\<rank>(p) > \<rank>(q)$, remove~$q$ from~$Q$ and insert it to~$P$ before~$p$,
\::otherwise (the ranks are equal, we need to propagate the carry):
\:::$\<carry>\=p$.
\:::Remove~$p$ from~$P$ and set $p\=$ the original successor of~$p$.
\:::While $\<rank>(q)=\<rank>(\<carry>)$:
\::::Remove~$q$ from~$Q$.
\::::$\<carry>\=\<join>(q, \<carry>)$.
\::::$q\=\<head>(Q)$.
\:::Insert~\<carry> before~$q$.
\:Update the suffix minima: Walk with~$p$ backwards to the head of~$P$:
\::$p'\=\<suffix\_min>$ of the successor of~$p$.
\::If $\<ckey>(p) < \<ckey>(p')$, set $\<suffix\_min>(p)\=p$.
\::Otherwise set $\<suffix\_min>(p)\=p'$.
\:Destroy the heap~$Q$.
\algout The merged heap~$P$.
\endalgo

\algn{Insertion of an~element to a~soft heap}
\algo
\algin A~heap~$H$ and a~new element~$x$.
\:Create a~new heap~$H'$ of the same parameters as~$H$. Let~$H'$ contain a~sole queue of rank~0,
  whose only vertex has the element~$x$ in its item list.
\:$\<Meld>(H,H')$.
\algout An~updated heap~$H$.
\endalgo

\para
So far, the mechanics of the soft heaps were almost identical to the binomial heaps
and the reader could rightfully yawn. The things are going to get interesting
now as we approach the parts where corruption of items takes place.

If all item lists contain at most one item equal to the \<ckey> of the particular
vertex, no information is lost and the heap order guarantees that the minimum item
of every queue stays in its root. We can however allow longer lists and let the items
stored in a~single list travel together between the vertices of the tree, still
represented by a~common \<ckey>. This data-structural analogue of car pooling will
allow the items to travel at a~faster rate, but as only a~single item can be equal
to the \<ckey>, all other items will be inevitably corrupted.

We of course have to be careful about the size of the lists, because we must avoid corrupting
too many items. We will control the growth according to the vertex ranks. Vertices
with rank at most~$r$ will always contain just a~single item. Above this level,
the higher is the rank, the longer list will be allowed.

\para
\em{Deletion of minimum} will be based on this principle. The minimum is easy to locate
--- we follow the \<suffix\_min> of the head of the heap to the queue with the minimum
\<ckey>. There we look inside the item list of the root of the queue. We remove the \em{last}
item from the list (we do not want the \<ckey> to change) and we return it as the minimum.
(It is not necessarily the real minimum of all items, but always the minimum of their
possibly corrupted values.)

If the list becomes empty, we \em{refill} it with items from the lower levels of the
same queue. This process can be best described recursively: We ask the left son to refill itself
(remember that the left son is always white, so there are currently no items there). If the new
\<ckey> of the left son is smaller than of the right son, we immediately move the left
son's list to its parent. Otherwise, we exchange the sons and move the list from the
new left son to the parent. This way we obey the heap order and at the same time we keep
the white left son free of items.

Ocassionally, we repeat this process once again and we concatenate the resulting lists
(we append the latter list to the former, using the smaller of the two \<ckey>'s). This
makes the lists grow longer and we want to do that roughly on every other level of the
tree. The exact condition will be that either the rank of the current vertex is odd,
or the difference in ranks between this vertex and its right son is at least two.

If refilling of the left son fails because there are no more items in that subtree
(we report this by setting its \<ckey> to $+\infty$), the current vertex is no longer
needed --- the items would just pass through it unmodified. We therefore want to
remove it. Instead of deleting it directly, we rather make it point to its former
grandsons and we remove the (now orhpaned) original son. This helps us to ensure
that both sons always keep the same rank, which will be useful for the analysis.

When all refilling is done, we update the suffix minima by walking from the current
queue to the head of the heap exactly as we did in the \<Meld> procedure.

Our only remaining worry is that the Rank invariant can be broken after the
refilling. When the leftmost path of the tree becomes too short, we just
\em{dismantle} the tree as already described and we meld the new trees back to
the heap. This is easier to handle when the item list at the root vertex is
empty. We will therefore move this check before the refilling of the root list.
It will turn out that we have enough time to always walk the leftmost path
completely, so no explicit counters are needed.

Let us translate these ideas to real (pseudo)code:

\algn{Deleting the minimum item from a~soft heap}
\algo
\algin A~soft heap~$H$.
\:Use \<suffix\_min> of the head queue of~$H$ to locate the queue~$q$ with the smallest \<ckey>.
\:Remove the final element~$x$ of the item list in the root of~$q$.
\:If the item list is empty:
\::Count the vertices on the leftmost path of~$q$.
\::If there are less than $\<rank>(q)$ of them:
\:::Remove~$q$ from the list of queues.
\:::Recalculate the suffix minima as in the \<Meld> procedure.
\:::Dismantle~$q$ and create a~heap~$H'$ holding the resulting trees.
\:::Meld them back: $\<Meld>(H,H')$.
\::Otherwise:
\:::Call \<Refill> on the root of~$q$.
\:::If $\<ckey>(q)=+\infty$ (no items left), remove the tree~$q$ from~$H$.
\:::Recalculate the suffix minima.
\algout The deleted minimum item~$x$ (possibly corrupted).
\endalgo

\algn{Refilling the item list of a~vertex}
\algo
\algin A~soft queue and its vertex~$v$ with an~empty item list.
\:Handle trivial cases: If~$v$ has no children or both have $\<ckey>=+\infty$,
  set $\<ckey>(v)$ to~$+\infty$ and return.
\:Let \<left> and~\<right> denote the respective sons of~$v$.
\:Recurse: call $\<Refill>(\<left>)$.
\:If $\<ckey>(\<left>) > \<ckey>(\<right>)$, swap the sons.
\:Move the item list from \<left> to~$v$ (implying $\<ckey>(v)=\<ckey>(\<left>)$).
\:If $\<rank>(v) > r$ and either $\<rank>(v)$ is odd or $\<rank>(v) > \<rank>(\<right>)+1$, recurse once more:
\::Repeat steps 3--4.
\::Append the item list from \<left> to the item list at~$v$.
\:Clean up. If $\<ckey>(\<right>) = +\infty$:
\::If $\<ckey>(\<left>) = +\infty$, unlink and discard both sons.
\::Otherwise relink the sons of \<left> to~$v$ and discard \<left>.
\algout A~modified soft queue.
\endalgo

\paran{Analysis of accuracy}
The description of the operations is now complete, so let us analyse their behavior
and verify that we have delivered what we promised --- first the accuracy of
the structure, then the time complexity of operations. In the whole analysis,
we will denote the total number of elements inserted during the history of the
structure by~$n$. We will also frequently take advantage of knowing that the
threshold~$r$ is even.

We start by bounding the sizes of the item lists.

\lemma
For every vertex~$v$ of a~soft queue, the size $\ell(v)$ of its item list
satisfies:
$$\ell(v) \le \max(1, 2^{\lceil \<rank>(v)/2 \rceil - r/2}).$$

\proof
Initially, all item lists contain at most one item, so the ineqality trivially
holds. Let us continue by induction. Melds can affect it only in the favorable
direction (they ocassionally move an~item list to a~vertex of a~higher rank)
and so do deletes (they only remove items from lists). The only potentially
dangerous place is the \<Refill> procedure.

Refilling sometimes just moves items upwards, which is safe, and sometimes it
joins two lists into one, which generally is not. When $\<rank>(v) \le r$,
no joining takes place and~$\ell(v)$ is still~1. Otherwise we join when either
$\<rank>(v)$ is odd or $\<rank>(w) < \<rank>(v)-1$ for any son~$w$ of~$v$ (remember
that both sons have the same rank). In both cases, $\lceil\<rank>(w)/2\rceil \le
\lceil\<rank>(v)/2\rceil - 1$. By the induction hypothesis, the size of each
of the two joined lists is at most $2^{\max(1,\lceil\<rank>(v)/2\rceil - 1 - r/2)}$,
so the new list has at most $2^{\lceil\<rank>(v)/2\rceil - r/2}$ items. (The maximum
has disappeared since $\<rank>(v)>r$ and therefore the desired bound is at least~2.)
\qed

We will now sum the sizes of the lists over all vertices containing corrupted items.

\lemma\id{shcorrlemma}%
At any given time, the heap contains at most~$n/2^{r-2}$ corrupted items.

\proof
We first prove an~auxiliary claim: The master trees of all queues contain at most~$n$
black vertices. This follows by induction: If no deletions have taken place,
there are exactly~$n$ black vertices, because insertion adds one black vertex and
melding preserves their number. A~deletion affects the master trees only when
dismantling takes place and then it only removes a~black vertex.

An~obvious upper bound on the number of corrupted items is the total size of item
lists in all vertices of rank greater than~$r$. We already know from the previous
lemma that the list sizes are limited by a~function of the ranks. A~complete tree
is obviously the worst case, so we will prove that this lemma holds for the master
tree of every queue in the heap. The actual trees can be much sparser, but the
above claim guarantees that the total size of the master trees is bounded by the
number of insertions properly.

So let us consider a~complete tree of~rank~$k$. It has exactly $2^{k-i}$ vertices
of rank~$i$ and each such vertex contains a~list of at most~$2^{\lceil i/2\rceil - r/2}$
items by the previous lemma. Summing over all ranks greater than~$r$, we get that
the total number of corrupted items in this tree is at most:
$$
\sum_{i=r+1}^k 2^{k-i}\cdot 2^{\lceil i/2\rceil - r/2}
= 2^{k-r/2} \cdot \sum_{i=r+1}^k 2^{\lceil i/2\rceil - i}
\le 2^{k-r/2+1/2} \cdot \sum_{i=r+1}^k 2^{-i/2}
\le 2^{k-r} \cdot \sum_{i=0}^\infty 2^{-i/2}.
$$
The sum of a~geometric series with quotient $2^{-1/2}$ is less than four, so the
last expression is less than $2^{k-r+2}$. Since the tree contains $n_k=2^k$ black vertices,
this makes less than $n_k/2^{r-2}$ corrupted items as we asserted.
\qed

\paran{Analysis of time complexity}
Now we will examine the amortized time complexity of the individual operations.
We will show that if we charge $\O(r)$ time against every element inserted, it is enough
to cover the cost of all other operations.

\FIXME{Pointer machine and yardsticks}

We take a~look at the melds first.

\lemma\id{shmeld}%
The amortized cost of a~meld is $\O(1)$, except for melds induced by dismantling
which take $\O(\<rank>(q))$, where $q$~is the queue to be dismantled.

\proof
The real cost of a~meld of heaps $P$ and~$Q$ is linear in the smaller of
their ranks, plus the time spent on carry propagation. The latter is easy to
dispose of: Every time there is a~carry, the total number of trees in all
heaps decreases by one. So it suffices to charge $\O(1)$ against creation of
a~tree. An~insert creates one tree, dismantling creates at most $\<rank>(q)$
trees, and all other operations alter only the internal structure of trees.

As for the $\O(\min(\<rank>(P),\<rank>(Q)))$ part, let us assume for a~while that
no dismantling ever takes place and consider the \df{meld forest.} It is a~forest
whose leaves correspond to the $n$~single-element heaps constructed by \<Insert>
and each internal vertex represents a~heap arisen from melding its sons. The left
son will be the one with the greater or equal rank. We therefore want to bound
the sum of ranks of all right sons.

For every right son, we will distribute the change for its rank~$k$ among all leaves
in its subtree. There are at least $2^k$ such leaves. No leaf ever receives the same
rank twice, because the ranks of right sons on every path from the root of the
tree to a~leaf are strictly decreasing. (This holds because melding of two heaps
always produces a~heap of a~strictly greater rank.) Hence at most~$n/2^k$
right sons have rank~$k$ and the total time charged against the leaves is bounded by:
$$
\sum_{k=0}^{\rm max. rank}k\cdot {n\over 2^k} \le n\cdot\sum_{k=0}^\infty {k\over 2^k} = 2n.
$$

Let us return dismantling to the game. When a~queue is dismantled, melding the parts
back to the heap takes $\O(\<rank>(q))$ time. We can therefore let the dismantling pay for it
and omit such induced melds from the meld forest. As the rank of a~heap is never increased
by induced melds, the above calculation is still a~proper upper bound on the cost
of the regular melds.
\qed

Before we estimate the time spent on deletions, we analyse the refills.

\lemma
Every invokation of the \<Refill> procedure takes time $\O(1)$ amortized.

\proof
When \<Refill> is called from the \<DeleteMin> operation, it recurses on a~subtree of the
queue. This subtree can be split to the ``lossless'' lower part (rank~$r$ and below)
and the upper part where list concatenation and thus also corruption takes place. Whenever
we visit the lower part during the recursion, we spend at worst $\O(r)$ time there.
We will prove that the total time spent in the upper parts during the whole life of the
data structure is $\O(n)$. Since each upper vertex can perform at most two calls to the lower
part, the total time spent in the lower parts is $\O(rn)$. All this can be prepaid by the
inserts.

Let us focus on the upper part. There are three possibilities of what can happen
when we visit a~vertex:

\itemize\ibull

\:We delete it: Every vertex deleted has to have been created at some time in the past.
New vertices are created only during inserts and melds (when joining two trees) and
we have already shown that these operations have constant amortized complexity. Then the
same must hold for deletions.

\:We recurse twice and concatenate the lists: The lists are disassembled only when
they reach the root of the tree, otherwise they are only concatenated. We can easily
model the situation by a~binary tree forest similar to the meld forest. There are~$n$
leaves and every internal vertex has outdegree two, so the total number of concatenations
is at most~$n$. Each of them can be performed in constant time as the list is doubly linked.

\:We recurse only once: This occurs only if the rank is even and the gap between the
rank of this vertex and its sons is equal to~1. It therefore cannot happen twice in a~row,
thus it is clearly dominated by the cost of the other possibilities.

\endlist
\>The total cost of all steps in the upper part is therefore $\O(n)$.
\qed

It remains to examine the rest of the \<DeleteMin> operation.

\lemma\id{shdelmin}%
Every \<DeleteMin> takes $\O(1)$ time amortized.

\proof
Aside from refilling, which is $\O(1)$ by the previous lemma, the \<DeleteMin>
takes care of the Rank invariant. This happens by checking the length of the leftmost
path and dismantling the tree if the length is too far from the tree's rank~$k$.
When the invariant is satisfied, the leftmost path is visited by the subsequent
call to \<Refill>, so we can account the check on the refilling.

When we are dismantling, we have to pay $\O(k)$ for the operation itself and
another $\O(k)$ for melding the trees back to the heap. Since we have produced at most
$k/2$ subtrees of distinct ranks, some subtree of rank $k/2$ or more must be missing.
Its master tree contained at least $2^{k/2}$ vertices which are now permanently gone
from the data structure, so we can charge the cost against them.
A~single vertex can participate in the master trees of several dismantlings, but their
ranks are always strictly increasing. By the same argument as in the proof of
Lemma \ref{shmeld} (complexity of \<Meld>), each vertex pays $\O(1)$.

We must not forget that \<DeleteMin> also has to recalculate the suffix minima.
In the worst case, it requires touching $k$~trees. Because of the Rank invariant,
this is linear in the size of the
leftmost path and therefore it can be also paid for by \<Refill>. (Incidentally,
this was the only place where we needed the invariant.)
\qed

Now we can put the bits together and laurel our effort with the following theorem:

\thmn{Performance of soft heaps, Chazelle \cite{chazelle:softheap}}
A~soft heap with error rate~$\varepsilon$ ($0<\varepsilon\le 1/2$) processes
a~sequence of operations starting with an~empty heap and containing $n$~\<Insert>s
in time $\O(n\log(1/\varepsilon))$ on the Pointer machine. At every moment, the
heap contains at most $\varepsilon n$ corrupted items.

\proof
We set the parameter~$r$ to~$2+2\lceil\log (1/\varepsilon)\rceil$. The rest follows
from the analysis above. By Lemma \ref{shcorrlemma}, there are always at most $n/2^{r-2}
\le \varepsilon n$ corrupted items in the heap. By Lemma \ref{shmeld}--\ref{shdelmin},
the time spent on all operations in the sequence can be paid for by charging $\O(r)$ time
against each \<Insert>, which yields the time bound.
\qed

\FIXME{Remark on optimality.}

\FIXME{Example of use: pivots.}



\endpart