[arch-dev-public] Adding multi arch, pkgver and/or pkgrel support in repo scripts

[Eric Bélanger]

On Thu, Feb 3, 2011 at 6:23 AM, Pierre Schmitz <pierre at archlinux.de> wrote:
> On Thu, 3 Feb 2011 05:05:11 -0500, Eric Bélanger wrote:
>> Hi,
>>
>> Lately, I've started to look into adding support in the dbscripts for
>> splitted packages with mixed arches (i686/x86_64 and any) and/or
>> different pkgver, pkgrel. Depending on my free time and how much time
>> it takes, it could take a while though. I'm just writing to make sure
>> no-one else is doing it so we don't duplicate work.  If anyone wants
>> to help out, let me know.  We'll split the work.
>
> I had put some thoughts into supporting "any" split packages. So far
> two points need to be looked into:
> 1) ensure all arches have the same pkgbuild

I suppose we could add a check for this. Diff might be adequate if we
ignore the PKGBUILd svn Id tags.

> 2) check_splitpkgs: get correct arch for svnnames
> Feel free to send me you ideas or some rough patches for a review.

I got rid of that function. It seemed to be used to make sure split
packages all have the same pkgver and pkgrel which is not longer
relevant. I suppose it could be readded later on with some of my
changes in it.  So far, I'm trying to get the correct logic so there's
some code duplication that might be moved to a function eventually.

more on my ideas below. I could send you my current work or post
patches here if other people are interested.

>
> But: I don't like supporting different pkgver or pkgrel for split
> packages. Imho this is a feature of makepkg that has issues by design:
> * Source packages with the same file name no longer point to the same
> packages

I haven't thought about the sourceballs script yet but there might be
a way to tell it to update the sourceballs.  Also, the reason we have
these sourceballs is to comply with licenses. As long as the upstream
sources are there, it shouldn't be a major problem if the PKGBUILD
inside them is a little bit out-of-date.  So it shouldn't be a stopper
IMO.

> * The repos subdir in svn (might) no longer contains the PKGBUILD that
> a package was built with

I don't see why or how this would happen. The trunk gets copied to the
repos subdir with archrelease.

> * Same issue with ABS which is no longer in sync with the actual binary
> packages

I believe that ABS use svn to gets its data. As I said above, it
shouldn't be affected.

> * Parsing the correct pkgver and pkgrel of a sub packages is between
> hard and nearly impossible
Possible and not very hard once you know how to do it. ;)  BTW, you
need to use the same procedure to get the correct arch for the split
package. Here's how it's done:

#get the default values of variables by sourcing PKGBUILD like below
or just do: sourcePKGBUILD
pkgname=($(. "PKGBUILD"; echo ${pkgname[@]}))
pkgver=$(. "$PKGBUILD"; echo "${pkgver}")
pkgrel=$(. "$PKGBUILD"; echo "${pkgrel}")
arch=($(. "$PKGBUILD"; echo ${arch[@]}))

for _pkgname in "${pkgname[@]}"; do

# check if the variables were overridden in the package functions. The
awk line gives you the individual package functions for each package.
This is what makes it possible. Then all what's left is just gepping
and sedding to get the data.
    unset newpkgver
    unset newpkgrel
    unset newarch
    newpkgver=$(awk "/package_$_pkgname\(/,/^}/" PKGBUILD |grep
pkgver= |sed 's/pkgver=//;s/ //g')
    newpkgrel=$(awk "/package_$_pkgname\(/,/^}/" PKGBUILD |grep
pkgrel= |sed 's/pkgrel=//;s/ //g')
    newarch=($(awk "/package_$_pkgname\(/,/^}/" PKGBUILD |grep arch=
|sed "s/arch=//;s/(//;s/)//;s/'//g"))

# If variables were overridden in package function, use that value
otherwise use the default one
    if [ -n "$newpkgver" ]; then
        _pkgver=$newpkgver
    else
        _pkgver=$pkgver
    fi

    if [ -n "$newpkgrel" ]; then
        _pkgrel=$newpkgrel
    else
        _pkgrel=$pkgrel
    fi

    if [ -n "$newarch" ]; then
        _arch=("${newarch[@]}")
    else
        _arch=("${arch[@]}")
    fi

# do stuff
echo $_pkgname-$_pkgver-$_pkgrel ${_arch[@]}

done

I've setup a local repo with svn tree to test this stuff. My test
PKGBUILKD produce the following packages:
diffarch-1-1-i686.pkg.tar.xz
diffarch-1-1-x86_64.pkg.tar.xz
diffarch-doc-1-2-any.pkg.tar.xz
libdiffarch-1.5-3-i686.pkg.tar.xz
libdiffarch-1.5-3-x86_64.pkg.tar.xz

As you can see, there quite a mix of different arch, pkgver and
pkgrel.  With the procedure described above, I was able to upload my
test package to my local staging directory and add it in my repo.  So
it seem that it's going to work.


> * We only have one build function; so treating split packages as one
> unit is best

I don't understang how it's related.  I still see them as one unit.

>
> In general this would make things more complicated and introduces quite
> some points of failure. So I don't think its worth to try to implement
> this atm. Which packages would benefit from that a lot?

- package that provide docs in a seperate package. Now, if you use a
split PKGBUILD, the docs need to be i686/x86_64.  If you want the docs
in an 'any' arch package, you need to use a seperate PKGBUILD wich is
more work for the maintainer.

- i18n packages, I've seen it in (open) libreoffice and koffice.
Sometime, they don't release new version of some of these i18n package
(or release them later) but the old version is still usable.
Currently, the package function of these languguage pack is commented
out in the PKGBUILD but the package is kept in the repo. So we end up
with a broken ABS and the integrity check complains. Support of multi
pkgver/pkrel in split package would fix that.

- Some split packages provides many packages, think about kdegames
which builds 40 packages.  If a single game needs to be fixed (missing
depends, etc), all the games needs to be packaged, uploaded to repo
and downloaded by users.  Support of multi pkgver/pkrel in split
package will enable us to only build upload the broken one.  makepkg
can build specific package in a split PKGBUILD. We could add similar
support in commitpkg.

>
> There are still some problems with split packages in dbscripts and
> those are complex enough; so we should be really careful and not make
> things even worse.

What problems? Are they on the bug tracker?

>
> Greetings,
>
> Pierre
>
> --
> Pierre Schmitz, https://users.archlinux.de/~pierre
>