A TCP/IP variation of -N UNZIP?
From
Mike Luther@1:117/3001 to
Jonathan de Boyne Pollard on Sun Aug 26 03:11:14 2001
I think, Johnathan, from what I've learned so far, that a part of what you've offered is correct.
JdBP> 1. Have some utility run on the remote end that creates a file
JdBP> containing the set of (binary) differences between
JdBP> two successive versions of the database file. Have
JdBP> a utility on the local end that takes such a
JdBP> "differences" file and a database and applies the
JdBP> former to the latter. Compress just the differences
JdBP> file for downloading. This is an analogue of the
JdBP> approach used by many large "open source" software
JdBP> projects, where instead of compressed snapshots of
JdBP> the entire source tree one can download compressed
JdBP> differences between one version and the next.
There appear to be two basic granularities of the project. One relates to perhaps hundreds of thousands of 'things' that are common across the entreprise. Even at that, what is suggested above, with modern equipment,is, I
think feasible.
JdBP> 2. Increase the granularity of the compressed bundle. Rather than
JdBP> having BUNDLE.ZIP containing A.TXT, B.TXT, and
JdBP> C.TXT, have A.ZIP, B.ZIP, and C.ZIP each containing
JdBP> a single file. Download each individual archive
JdBP> file if its datestamp indicates that it has been
JdBP> updated. This approach presumes that, as your
JdBP> original message implied, your database is a set of
JdBP> multiple individual files rather than a single
JdBP> humungous BUNDLE.DAT file with a complex internal
JdBP> structure.
And the above is exactly what is needed, simply for storage needs.
I'm thinking carefully on how it may be possible to implement the above.
The other issue I face is, I think, more complicated.
A given box generates perhaps a hundred transactions a day. The field of action in the market spans, even at present, over a million and a half boxes. Thus the transaction processing load which has to be handled and stored on a daily basis is .. realistically, over a million transaction processing events a
day. Each of them has to be recoverable and massaged as to what the disposition of them was, at Empire Central, at least ONCE at the originating embedded boxlette end. From that point on any one of them actually has to be recoverable, on demand, for 50 years or longer, in some cases we already know about.
Mike @ 1:117/3001
--- Maximus/2 3.01
* Origin: Ziplog Public Port (1:117/3001)