Internals documentation for buffindexed

[eagle]

We should have internals documentation for buffindexed and a detailed document of the on-disk format. Here are some notes from Katsuhiro Kondou:

Buffindexed has two functions.

1. block manager
2. overview data/index manager

Block manager:

All buffers are divided into 8KB block, and there is a bit map like cnfs. The block can be used for both overview data and index. Block manager does not care the usage of the block. It just gives a free block and frees it if required.

Overview data/index manager:

There is group.index which gives the pointer to each newsgroup info, hash of the group, the number of article in the group, hi/lo mark and the flag. That pointer includes where are base index block, last index block, current index block, the next block of current index and current overview data block. Index block includes a series of token, arrival time, where the overview data block is, offset for the data and the length of data for each article. Each block allocated by block manager is used for one group and never shared with other newsgroups. Current overview manager assumes all index rooms are allocated for all articles between low and high mark. When expiry, overview manager recreate the information into new block by checking article existence. It frees all old blocks and replaces all info in group.index for the group.