[pacman-dev] [PATCH] Add deltas to repository database.
Dan McGee
dpmcgee at gmail.com
Sun Oct 7 21:29:34 EDT 2007
On 10/5/07, Nathan Jones <nathanj at insightbb.com> wrote:
> This patch adds delta information to the repository database along with
> the other information about a package. This would be useful if two or
> more package versions have been built since a user upgrades.
>
> I don't think it would be too difficult to create an algorithm to decide
> which delta files, if any, would be downloaded and applied.
>
> Any comments?
>
> Sample output:
>
> $ ls -l *.pkg.tar.gz *.delta
> -rw-r--r-- 1 nathanj users 1045940 2007-05-12 21:03 pan-devel-0.129-1-i686.pkg.tar.gz
> -rw-r--r-- 1 nathanj users 583339 2007-10-05 18:38 pan-devel-0.129-1_to_0.130-1-i686.delta
> -rw-r--r-- 1 nathanj users 1044271 2007-05-19 13:17 pan-devel-0.130-1-i686.pkg.tar.gz
> -rw-r--r-- 1 nathanj users 425727 2007-10-05 18:38 pan-devel-0.130-1_to_0.131-1-i686.delta
> -rw-r--r-- 1 nathanj users 1055063 2007-05-26 18:40 pan-devel-0.131-1-i686.pkg.tar.gz
> -rw-r--r-- 1 nathanj users 680644 2007-10-05 18:38 pan-devel-0.131-1_to_0.132-1-i686.delta
> -rw-r--r-- 1 nathanj users 1025859 2007-08-15 22:03 pan-devel-0.132-1-i686.pkg.tar.gz
> -rw-r--r-- 1 nathanj users 1792 2007-10-05 18:38 pan-devel-0.132-1_to_0.132-2-i686.delta
> -rw-r--r-- 1 nathanj users 1025826 2007-08-01 17:04 pan-devel-0.132-2-i686.pkg.tar.gz
> $ ~/code/pacman/scripts/repo-add repo.db.tar.gz pan-devel-0.132-2-i686.pkg.tar.gz *.delta
> ==> Extracting database to a temporary location...
> ==> Adding package 'pan-devel-0.132-2-i686.pkg.tar.gz'
> -> Removing existing package 'pan-devel-0.132-2'...
> -> Creating 'desc' db entry...
> -> Computing md5 checksums...
> -> Creating 'depends' db entry...
> ==> Adding delta 'pan-devel-0.129-1_to_0.130-1-i686.delta'
> ==> Adding delta 'pan-devel-0.130-1_to_0.131-1-i686.delta'
> ==> Adding delta 'pan-devel-0.131-1_to_0.132-1-i686.delta'
> ==> Adding delta 'pan-devel-0.132-1_to_0.132-2-i686.delta'
> ==> Creating updated database file /home/nathanj/pkgs/pan-devel/repo.db.tar.gz
> $ tar zxOf repo.db.tar.gz pan-devel-0.132-2/deltas
> %DELTAS%
> 0.129-1 0.130-1 583339
> 0.130-1 0.131-1 425727
> 0.131-1 0.132-1 680644
> 0.132-1 0.132-2 1792
>
> Signed-off-by: Nathan Jones <nathanj at insightbb.com>
> ---
> scripts/repo-add.sh.in | 64 +++++++++++++++++++++++++++++++++++++++++++++++-
> 1 files changed, 63 insertions(+), 1 deletions(-)
>
> diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in
> index cb741d7..4a7949e 100644
> --- a/scripts/repo-add.sh.in
> +++ b/scripts/repo-add.sh.in
> @@ -97,6 +97,61 @@ write_list_entry() {
> fi
> }
>
> +# write a delta entry to the pacman database
> +# arg1 - path to delta
> +db_write_delta()
> +{
> + # blank out all variables and set deltafile
> + local deltafile=$(readlink -f "$1")
> + local pkgname fromver tover arch csize
> +
> + pkgname="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\1/')"
> + fromver="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\2/')"
> + tover="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\3/')"
> + arch="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\4/')"
> +
> + # get size of delta
> + csize=$(du -b -L "$deltafile" | cut -f 1)
> +
> + pushd "$gstmpdir" 2>&1 >/dev/null
> +
> + # ensure $pkgname and $pkgver variables were found
> + if [ -z "$pkgname" -o -z "$fromver" ]; then
> + error "$(gettext "Invalid delta file '%s'.")" "$deltafile"
> + popd 2>&1 >/dev/null
> + return 1
> + fi
> +
> + # cd into the package's directory
> + local existing
> + local gotpackage=f
> + for existing in *; do
> + if [ "${existing%-*-*}" = "$pkgname" ]; then
> + cd "$existing"
> + gotpackage=t
> + break
> + fi
> + done
> +
> + # exit out if the package's directory does not exist
> + if [ "$gotpackage" == "f" ]; then
> + error "$(gettext "Cannot find directory for '%s'.")" "$pkgname"
> + popd 2>&1 >/dev/null
> + return 1
> + fi
> +
> + # create deltas file if it doesn't exist
> + if [ ! -f "deltas" ]; then
> + echo -e "%DELTAS%" >>deltas
> + fi
> +
> + # add the entry for this delta file
> + echo -e "$fromver $tover $csize" >>deltas
> +
> + popd 2>&1 >/dev/null
> +} # end db_write_delta
> +
> +
> # write an entry to the pacman database
> # arg1 - path to package
> db_write_entry()
> @@ -242,7 +297,14 @@ for arg in "$@"; do
> fi
> else
> if [ -f "$arg" ]; then
> - if ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then
> + # see if extension is .delta ... there's gotta be a better way
> + if [ "${arg%.delta}" != "$arg" ]; then
> + msg "$(gettext "Adding delta '%s'")" "$arg"
> +
> + if db_write_delta "$arg"; then
> + success=1
> + fi
> + elif ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then
> error "$(gettext "'%s' is not a package file, skipping")" "$arg"
> else
> msg "$(gettext "Adding package '%s'")" "$arg"
> --
> 1.5.3.4
This idea seems pretty smart to me. A few comments/questions:
1. We probably need a way to remove 'old' deltas. Of course, I don't
know what to define as old (probably retain no more than 5 deltas or
something).
2. Although sed magic is cool, there has to be a better way to get to
that info (pkigname, oldver, etc.)
3. Could delta addition be automatic? Right now the entries would be
deleted every time you upgrade a package in the database (since the
old entry is removed). Perhaps on an add, we should save the old delta
entries, copy them over to the new package, and then add any new
deltas that we find?
This is meant to be constructive criticism- please don't let that
discourage you, I like this idea.
-Dan
More information about the pacman-dev
mailing list