Thread

Posted on Mon Feb 7 21:48:36 2005 by rjbs
CPAN::Mini of the Future!

I'm working on sketches and notes and angry muttering on the next iteration of CPAN::Mini. I want to keep it fairly simple, but add enough sophistication that it's easier to subclass and plug in. Here are a few things that will be happening:

There will be a "Source" class that will represent the remote CPAN mirror. It will provide a list of modules or distributions and the information needed to retrieve them.

There will be a Mirror or Processor class (or both) that will perform the mirroring from the source, possibly doing extra work via plugins.

There will be (either in the core dist or as an extension) a Source multiplexer that will let you mirror multiple sources.

Output will include a new index (02packages etc) built from the actual mirroring that is performed. Modules you skip will not be listed, for example.

There will be more tests.

I will post more details as they become more than squiggles on my whiteboard.

Direct Responses: 99 | 153 | 170 | 171 | Write a response
Posted on Sat Feb 12 17:17:27 2005 by jd in response to 44
Re: CPAN::Mini of the Future!

Sounds terrific. Not only do I want to perform limited mirroring (e.g., current packages only, vice all revisions) but then easily update yet another mirror on a disconnected private network.

Updating this disconnected network mirror would require a synchronization operation from my connected mirror, probably conveyed via CD or DVD. To accomplish this, my connected mirror needs to keep state since the last sync-to-disconnected mirror, to be able to determine the diff set.

Anyway, I'm hoping to make this a fairly easy and regular operation. And, of course, apply it to more that just CPAN :)

Off to check out CPAN-mini more closely

Write a response
Posted on Sat Feb 26 21:35:02 2005 by dpavlin in response to 44
Re: CPAN::Mini of the Future!

I also started from same code, and made some changes which might be useful to CPAN::Mini. Take a look at: http://svn.rot13.org/~dpavlin/svnweb/index.cgi/perl/log/trunk/mirror_cpan.pl

I thinks that most useful feature is creation of (optinally compressed) readme file for each module. That allows me to use readme command from cpan shell (which I do a lot :-)

Are you interesed in this change so that I can stop maintaning my own version?

Write a response
Posted on Wed Mar 2 04:53:59 2005 by itub in response to 44
Re: CPAN::Mini of the Future!

This sounds like it is contemplated in your plans already, but here's an example of something I wanted to do:

I subclassed CPAN::Mini and I wanted to do some processing on each new file as it was fetched. The way I ended up doing it was copying and pasting mirror_file() into my subclass, and then adding a line after if ( $status == RC_OK ) saying $self->post_mirror($local_file) and doing all the postprocessing in post_mirror(). It would have been more convenient if the base class already had that hook, with an empty post_mirror() by default.

Write a response
Posted on Wed Mar 2 10:55:23 2005 by rrwo in response to 44
Re: CPAN::Mini of the Future!

I would like to see the filters redone so that a subclassed filter (which should be a public method) can "rewrite" the entry rather than simply give a thumbs up or down on it. The reason is that I could update CPAN::Mini::Tested to download the most recent version of a module that has passed tests, rather than simply not download a module if the most recent one has not passed a test. (It currently has the side-effect that when a new module is uploaded, it will delete the latest version if the newest one has not yet been tested.)

Write a response