Discussion:
[Pdl-porters] PDL & Stratopan
Diab Jerius
2014-09-10 14:23:56 UTC
Permalink
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).

Last night Jeffrey Thalhammer remotely gave a talk about Stratopan at
the boston.pm meeting which inspired me to think about how we could
solve problems like this.

For those not familiar with Stratopan, it's essentially a cloud based
Pinto repository. Pinto is a tool for maintaining independent,
version controlled, subsets of CPAN. As far as CPAN clients are
concerned, Stratopan looks just like CPAN, just with a different URL,
and, importantly, with a curated set of known good dependencies.

How does this fit in with PDL? As with any large codebase, PDL
depends upon various CPAN modules; installation from CPAN is subject
to whatever the current state of those modules on CPAN. In this case,
that state is broken. What Stratopan offers is the ability to tell
users,

Point your cpan client at our Stratopan repository,
and we'll guarantee that you'll be able to install PDL.

We'd no longer be at the mercy of CPAN breakage.

My suggestion is that we create an official PDL repository on
Stratopan. When a formal production release is cut, a set of "known
good" dependencies would be established as a Pinto stack (essentially
a branch) on Stratopan. If need be, patched versions of modules could
be installed there. Installation instructions could point to that
stack as an alternative to CPAN.

CPAN would of course continue to be used as a testbed for development
releases (and for those who don't mind fixing things when they break).

Over the years I've dealt with similar issues with CPAN modules using
the distroprefs functionality in CPAN.pm, which can automatically
patch modules pulled from CPAN. Unfortunately this requires somewhat
obscure knowledge about CPAN.pm's workings, and more importantly,
isn't supported by cpanm.

Stratopan isn't a panacea, of course. There will always be ways of
breaking PDL after a clean install using it. For example, there's
still the chance that the user may update a dependency from CPAN which
introduces new run-time behavior which breaks PDL. We could, however,
promise that if PDL is installed into it's own local lib, with known
good dependencies, that that will always work in isolation from
whatever else the user does with their Perl environment.

Diab
Ed .
2014-09-10 14:42:40 UTC
Permalink
I would suggest that what you actually need is less than that; once
ExtUtils::MakeMaker can do version ranges in its prerequisites (coming
soonish), you could adjust PDL's build system to do that.

I'll predict I make EUMM do version ranges before PDL's build system gets
updated.

-----Original Message-----
From: Diab Jerius
Sent: Wednesday, September 10, 2014 3:23 PM
To: pdl-***@jach.hawaii.edu
Subject: [Pdl-porters] PDL & Stratopan

A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).

Last night Jeffrey Thalhammer remotely gave a talk about Stratopan at
the boston.pm meeting which inspired me to think about how we could
solve problems like this.

For those not familiar with Stratopan, it's essentially a cloud based
Pinto repository. Pinto is a tool for maintaining independent,
version controlled, subsets of CPAN. As far as CPAN clients are
concerned, Stratopan looks just like CPAN, just with a different URL,
and, importantly, with a curated set of known good dependencies.

How does this fit in with PDL? As with any large codebase, PDL
depends upon various CPAN modules; installation from CPAN is subject
to whatever the current state of those modules on CPAN. In this case,
that state is broken. What Stratopan offers is the ability to tell
users,

Point your cpan client at our Stratopan repository,
and we'll guarantee that you'll be able to install PDL.

We'd no longer be at the mercy of CPAN breakage.

My suggestion is that we create an official PDL repository on
Stratopan. When a formal production release is cut, a set of "known
good" dependencies would be established as a Pinto stack (essentially
a branch) on Stratopan. If need be, patched versions of modules could
be installed there. Installation instructions could point to that
stack as an alternative to CPAN.

CPAN would of course continue to be used as a testbed for development
releases (and for those who don't mind fixing things when they break).

Over the years I've dealt with similar issues with CPAN modules using
the distroprefs functionality in CPAN.pm, which can automatically
patch modules pulled from CPAN. Unfortunately this requires somewhat
obscure knowledge about CPAN.pm's workings, and more importantly,
isn't supported by cpanm.

Stratopan isn't a panacea, of course. There will always be ways of
breaking PDL after a clean install using it. For example, there's
still the chance that the user may update a dependency from CPAN which
introduces new run-time behavior which breaks PDL. We could, however,
promise that if PDL is installed into it's own local lib, with known
good dependencies, that that will always work in isolation from
whatever else the user does with their Perl environment.

Diab
Diab Jerius
2014-09-10 15:11:40 UTC
Permalink
That'll be a truly great addition to EUMM, but it won't help if a
dependency isn't as strict about its dependencies as the top-level
module.

If the dependency chain is

PDL -> Module A -> Module B

and Module B is broken on CPAN, it doesn't matter if PDL is strict
about versions for Module A if that isn't strict about versions for
Module B.

The only way to control the entire dependency chain is to supply all
of the dependencies.
Post by Ed .
I would suggest that what you actually need is less than that; once
ExtUtils::MakeMaker can do version ranges in its prerequisites (coming
soonish), you could adjust PDL's build system to do that.
I'll predict I make EUMM do version ranges before PDL's build system gets
updated.
-----Original Message----- From: Diab Jerius
Sent: Wednesday, September 10, 2014 3:23 PM
Subject: [Pdl-porters] PDL & Stratopan
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
Last night Jeffrey Thalhammer remotely gave a talk about Stratopan at
the boston.pm meeting which inspired me to think about how we could
solve problems like this.
For those not familiar with Stratopan, it's essentially a cloud based
Pinto repository. Pinto is a tool for maintaining independent,
version controlled, subsets of CPAN. As far as CPAN clients are
concerned, Stratopan looks just like CPAN, just with a different URL,
and, importantly, with a curated set of known good dependencies.
How does this fit in with PDL? As with any large codebase, PDL
depends upon various CPAN modules; installation from CPAN is subject
to whatever the current state of those modules on CPAN. In this case,
that state is broken. What Stratopan offers is the ability to tell
users,
Point your cpan client at our Stratopan repository,
and we'll guarantee that you'll be able to install PDL.
We'd no longer be at the mercy of CPAN breakage.
My suggestion is that we create an official PDL repository on
Stratopan. When a formal production release is cut, a set of "known
good" dependencies would be established as a Pinto stack (essentially
a branch) on Stratopan. If need be, patched versions of modules could
be installed there. Installation instructions could point to that
stack as an alternative to CPAN.
CPAN would of course continue to be used as a testbed for development
releases (and for those who don't mind fixing things when they break).
Over the years I've dealt with similar issues with CPAN modules using
the distroprefs functionality in CPAN.pm, which can automatically
patch modules pulled from CPAN. Unfortunately this requires somewhat
obscure knowledge about CPAN.pm's workings, and more importantly,
isn't supported by cpanm.
Stratopan isn't a panacea, of course. There will always be ways of
breaking PDL after a clean install using it. For example, there's
still the chance that the user may update a dependency from CPAN which
introduces new run-time behavior which breaks PDL. We could, however,
promise that if PDL is installed into it's own local lib, with known
good dependencies, that that will always work in isolation from
whatever else the user does with their Perl environment.
Diab
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
Chris Marshall
2014-09-10 15:18:29 UTC
Permalink
A Bundle::PDL could include all the modules needed, including
dependencies.

--Chris
Post by Diab Jerius
That'll be a truly great addition to EUMM, but it won't help if a
dependency isn't as strict about its dependencies as the top-level
module.
If the dependency chain is
PDL -> Module A -> Module B
and Module B is broken on CPAN, it doesn't matter if PDL is strict
about versions for Module A if that isn't strict about versions for
Module B.
The only way to control the entire dependency chain is to supply all
of the dependencies.
Post by Ed .
I would suggest that what you actually need is less than that; once
ExtUtils::MakeMaker can do version ranges in its prerequisites (coming
soonish), you could adjust PDL's build system to do that.
I'll predict I make EUMM do version ranges before PDL's build system gets
updated.
-----Original Message----- From: Diab Jerius
Sent: Wednesday, September 10, 2014 3:23 PM
Subject: [Pdl-porters] PDL & Stratopan
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
David Mertens
2014-09-10 15:53:38 UTC
Permalink
And by the way, the latest Module::Compile was released in mid-August. I filed
a bug report <https://github.com/ingydotnet/module-compile-pm/issues/1>
with Module::Compile on this very issue a couple of weeks ago in late
august, hoping Ingy would be able to correct it and spin a quick turn
around. So far, no response, except a pull request with a potential fix.
Hopefully Ingy can fix this soon.

In the meantime, I think that having an official Stratopan PDL address is
*brilliant*.

David
Post by Chris Marshall
A Bundle::PDL could include all the modules needed, including
dependencies.
--Chris
Post by Diab Jerius
That'll be a truly great addition to EUMM, but it won't help if a
dependency isn't as strict about its dependencies as the top-level
module.
If the dependency chain is
PDL -> Module A -> Module B
and Module B is broken on CPAN, it doesn't matter if PDL is strict
about versions for Module A if that isn't strict about versions for
Module B.
The only way to control the entire dependency chain is to supply all
of the dependencies.
Post by Ed .
I would suggest that what you actually need is less than that; once
ExtUtils::MakeMaker can do version ranges in its prerequisites (coming
soonish), you could adjust PDL's build system to do that.
I'll predict I make EUMM do version ranges before PDL's build system
gets
Post by Diab Jerius
Post by Ed .
updated.
-----Original Message----- From: Diab Jerius
Sent: Wednesday, September 10, 2014 3:23 PM
Subject: [Pdl-porters] PDL & Stratopan
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan
Chris Marshall
2014-09-10 15:14:32 UTC
Permalink
One of the directions we've been steering toward with PDL distribution
is to spin off the non-core modules into their own distributions and to
use a CPAN bundle to install the "whole" PDL distribution that would
be equivalent to the kitchen sink install we have now.

Ed, do you know if the Bundle installs do a specific version or the
same as EUMM with just lower bounds?

--Chris
Post by Ed .
I would suggest that what you actually need is less than that; once
ExtUtils::MakeMaker can do version ranges in its prerequisites (coming
soonish), you could adjust PDL's build system to do that.
I'll predict I make EUMM do version ranges before PDL's build system gets
updated.
-----Original Message----- From: Diab Jerius
Sent: Wednesday, September 10, 2014 3:23 PM
Subject: [Pdl-porters] PDL & Stratopan
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
Last night Jeffrey Thalhammer remotely gave a talk about Stratopan at
the boston.pm meeting which inspired me to think about how we could
solve problems like this.
For those not familiar with Stratopan, it's essentially a cloud based
Pinto repository. Pinto is a tool for maintaining independent,
version controlled, subsets of CPAN. As far as CPAN clients are
concerned, Stratopan looks just like CPAN, just with a different URL,
and, importantly, with a curated set of known good dependencies.
How does this fit in with PDL? As with any large codebase, PDL
depends upon various CPAN modules; installation from CPAN is subject
to whatever the current state of those modules on CPAN. In this case,
that state is broken. What Stratopan offers is the ability to tell
users,
Point your cpan client at our Stratopan repository,
and we'll guarantee that you'll be able to install PDL.
We'd no longer be at the mercy of CPAN breakage.
My suggestion is that we create an official PDL repository on
Stratopan. When a formal production release is cut, a set of "known
good" dependencies would be established as a Pinto stack (essentially
a branch) on Stratopan. If need be, patched versions of modules could
be installed there. Installation instructions could point to that
stack as an alternative to CPAN.
CPAN would of course continue to be used as a testbed for development
releases (and for those who don't mind fixing things when they break).
Over the years I've dealt with similar issues with CPAN modules using
the distroprefs functionality in CPAN.pm, which can automatically
patch modules pulled from CPAN. Unfortunately this requires somewhat
obscure knowledge about CPAN.pm's workings, and more importantly,
isn't supported by cpanm.
Stratopan isn't a panacea, of course. There will always be ways of
breaking PDL after a clean install using it. For example, there's
still the chance that the user may update a dependency from CPAN which
introduces new run-time behavior which breaks PDL. We could, however,
promise that if PDL is installed into it's own local lib, with known
good dependencies, that that will always work in isolation from
whatever else the user does with their Perl environment.
Diab
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
Chris Marshall
2014-09-10 15:16:17 UTC
Permalink
Please open a bug ticket for this. At the least, the Known_Problems
file should be updated. Also, since the Module::Compile work looks
to need a reboot (various portability reasons) it might make sense to
remove that as a dependency or to add Module::Install as a work-around.

--Chris
Post by Diab Jerius
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
Diab Jerius
2014-09-17 20:18:11 UTC
Permalink
Belatedly done.

https://sourceforge.net/p/pdl/bugs/365/
Post by Chris Marshall
Please open a bug ticket for this. At the least, the Known_Problems
file should be updated. Also, since the Module::Compile work looks
to need a reboot (various portability reasons) it might make sense to
remove that as a dependency or to add Module::Install as a work-around.
--Chris
Post by Diab Jerius
A few days ago I was briefly stymied by the inability to perform a
clean install of PDL 2.007 from CPAN, due to a recent release of
Module::Compile missing a dependency on Module::Install
(https://github.com/ingydotnet/module-compile-pm/issues/1).
Loading...