[aur-dev] [AUR2] Need some feedback

[eliott]

> First off I'd like some feedback on the RPC interface. I know that AUR
> started work on using JSON, and in my mind it's the best contender.
> Django has inbuilt support for serialising to XML, JSON and YAML, so I
> could easily use either of those. I'm mainly interested in the user
> end of this - the AUR frontend (yaourt, etc.) developers. I haven't
> found anything to parse JSON or XML easily from bash, so I don't think
> it would be easy to implement either way in yaourt and other
> frontends. There's a Perl CPAN module[1] which may be easily used to
> quickly parse the data and allow the rest of the work to be done via
> bash, which, I guess, would be a decent compromise.
> In my opinion XML is just way too heavy and has too many unnecessary
> features for what we're going to use it for. YAML seems to be about
> the same, but JSON is a nice data oriented RPC that can be quickly
> parsed. If all goes well and we choose JSON, the interace will be
> compatible with AUR's, though I haven't looked int othe code much yet.

I myself am *very* fond of json data exchange. Simple http get
requests returning json formatted data is lightweight, fairly restful,
and damn easy to scale (even cache/proxy).

I am not a huge fan of xml for data exchange, and consider it more of
a ... display language. *shrug*
json is a subset of yaml. I like yaml just fine, but it doesn't seem
as well supported by most things web oriented as json (raw javascript,
javascript libraries, other language libraries, etc). Other than that,
I do like yaml.

As far as bash processing, since many languages have decent json
libraries (like python), it would be fairly simple to have a python
helper script (or some simple C binary) that spews out the data in
some well defined format that people can sed/awk/grep to joy.
SOME_DATA=`wget http://masterful.aur.site/json?q=my+awesome+app+query
-O - | /usr/sbin/return-joy_aur-helper`

> On the wiki[1] someone asked to have versioned dependencies. Currently
> I've only implemented listing of dependencies and don't even track the
> version requirements. I'm not certain that this functionality is
> necessary at all. It has very little use, and the only use cases I can
> think of are validation the package, so that it's compatible with the
> dependencies (ie. disallowing to upload a package that depends on an
> older version of a package in AUR), and simply being notified that the
> versions don't match. Both of these, in my opinion, are pointless.

I agree. Largely pointless for the vast majority.
Edge cases shouldn't be the focus of development.

> As with versioned dependencies, someone asked for the ability to
> search for outdated packages.

Outdated packages in the aur?
Right now there is only a comment system for people knowing if a
package is out of date. This doesn't yield useful searchable criteria.

> Another request was for regexp searching.

Regex searching seems a bit... excessive.
mysql FULLTEXT style searching seems 'good enough' for the aur. I just
can't think of a scenario where I would need to bust out with regex to
find the name of a package I was looking for. One of the main
strengths of regex is the capture ability, and that just isn't a
usecase for raw searching.

Like I said, for the aur it seems like an over engineering style
feature request just for coolness sake.

> Now the biggest issue I have is integrating an SCM. This should
> probably go in a separate thread, but I may as well discuss it here. I
> don't know how AUR or the main site handle it, I'd have to look into
> that, but I would really like to integrate an SCM backend for handling
> files. Initially I had the idea for the interaction to go both ways; a
> user could simply "git commit -a -m 'bumped version'; git push" and
> the everything would be updated in the database, but also for the
> ability to do it via the web interface like it works now. Dan pointed
> out that it would be hard to restrict access for the maintainers to
> the various repositories/modules (or however the repository would be
> split). The whole idea could generally raise many problems, so it may
> be better to go for an easier solution and have it only work one way,
> ie. via the web interface. It appears the TUs use CVS directly to
> upload packages, so they may be a but frustrated with that move.
> There's also the issue of which SCM to use. It seems the main site is
> leaning towards converting to SVN (and rsync) from CVS, but git is
> also a contender. Optimally only portions of the repositories could be
> checked out, so instead of checking out the whole repository, as it
> currently is with abs, you'd be able to just have the packages that
> you maintain. I don't know of any SCM that implements this well
> though.
> I would really love some feedback on any ways to implement this, which
> SCM to use, etc. But I don't want discussions to go on for ever like
> they are with the main site conversion.

The arch dev team will be using subversion. Aaron decided enough
discussion was enough, and we can always change later if need be. No
sense sitting in mud because you don't know if you want to wear pants
or shorts once you get clean.

I don't think general scm submission access makes any sense at all for the aur.
Now, for backend storage, *maybe*, but that is debatable.

I think the AUR needs to get 'back to roots', and focus on doing the
job it needs to do, and doing it well.
I think extraneous features that people want to add to AUR2
development should be punted, or forgotten altogether. Focus on core
use and mainstream features.

** Massive Sidebar **
As a sidenote, I think the TU devs (community) should have a more
custom arch-dev like interface (cli tools, flag out of date interface,
etc), and not really be included in the aur at all. Alot of the TU
functionality was somewhat 'bolted on', and wasn't ever really a good
fit for the real core use cases of the AUR.