Kodewerx

So I was digging through my doc, and I ran into a text file I had tossed in there a couple of months ago.

The contents of it:

http://pastebin.com/nB5vck7Z

I personally think it's simple and genius.

But at the moment I just don't have the time to mess with the idea.

C&C yo.

_________________

-> Programmering

The only things I like about the UPS format is that it supports files of arbitrary size and reversing patches ("unpatching" or "undoing" a previously applied patch). Other than that, it's still just IPS in sheep's clothing.

Re: encryption - No. See: viewtopic.php?p=76913#p76913

Re: compression - Negligible. Compression is "nice to have" but unless you're dealing with bandwidth or storage space sensitivity, compression is just a means to add unnecessary complexity.


What's really needed in a patch format is the following:

1. Simple spec
2. Arbitrary file and patch size limits
3. Binary deltas

Everything else (checksum/hash verification, compression, embedded descriptions, tags, credentials, etc.) is firmly in the "nice to have but not really important" category.

Some examples of currently available [open source] "binary delta patching" software:

• xdelta
• bsdiff
• bdiff

To my knowledge, these all cover points 2 and 3 above. But point #1? Hmmm...

_________________
I have to return some video tapes.

Feed me a stray cat.

Any format I come up with would at least need extendability if not including the features I want because I have different needs.

_________________

I went ahead and did this.

Hextator's Doc/$Development/@Misc/New Patching Format

Personally I think it's glorious, but whatever.

I hated the VLI format you were thinking about for the SRDP and went and made my own. It's moderately efficient and definitely simple, but nothing revolutionary.

_________________

Or so you keep telling me over AIM continuously since you started work on this all of a day or so ago?

I don't see anything about delta patching?

Until it can create/apply a minimal patch against an original and modified file system*, it's no better than IPS/UPS.


Also, an alternative to VLI is arbitrary precision numbers. For example, these functions in GMP would be perfect for getting/setting numbers of unknown size: http://gmplib.org/manual/Integer-Import-and-Export.html

The only problem I can think of is a portability issue: size_t (for the count variable) is the native word size on the processor; 32-bit on 32-bit CPUs, 64-bit on 64-bit CPUs. To make it truly portable, you would have to stick to the least common denominator, which, according to the C99 spec, is 16-bits: http://en.wikipedia.org/wiki/Size_t

Just use a size of 1, and you have a binary format like this: <uint16_t count>,<uint8_t array[count]>. "Sadly" that limits you to a number between 8 bits and 524,280 bits.


* The "file system" may be a disk image, ISO, NDS ROM, whatever; the point is, once a file in the system grows or shrinks (and the file system is "defragmented" or "rebuilt"), file data moves. A simple byte-by-byte compare of such a modification will typically lead to a patch that contains most (if not all) of the modified file, rather than a minimal series of patches (changes, additions, subtractions, deltas/moved data, etc.)

_________________
I have to return some video tapes.

Feed me a stray cat.

It needs revision anyway. I had in the back of my head this eerie feeling that storing offsets was "too easy" and then I was reminded by the UPS spec (which I finally found) that I should be storing relative offsets instead of absolute ones to truly take advantage of the VLIs.

Yes, this is only a minor improvement over UPS at a glance, until you realize just how annoying the UPS format's integrity specification is to use in practice.

The spec and poc have been updated to use relative offsets and the source code for the poc now has more comments.

_________________