[Pdl-porters] longlong<->double bug fix

[Chris Marshall]

I would like to wrap up the 64bit index support for PDL-2.008
by fixing the longlong to double type coercion in the PDL Core.
This would entail changing the default any-type PDL value
from double to something else.

Similarly, the badvalue would need to be in an appropriate
type (arguably the type of the piddle itself) but maybe just
the same as the new Core type for symmetry.

It seems that a reasonable option might be a union type of
all the elemental PDL types plus pointers. Which types are
to be elemental could be refined later but we could start
with the current PDL types. The motivation for the pointer
is that we would use pass by reference to handle the most
general cases (conveniently, it also allows for all the perl
SV*, AV*,... as well).

For 2.008 there would be just enough "glue" to resolve the
lingering issues for longlong and 64bit PDL_Indx data. After
that release, I suggest we switch from the master branch to
working with the newly cleaned up core that Craig has been
working on.

Thoughts?
Chris

[Burke, Douglas]

Post by Chris Marshall
I would like to wrap up the 64bit index support for PDL-2.008
by fixing the longlong to double type coercion in the PDL Core.
This would entail changing the default any-type PDL value
from double to something else.
Similarly, the badvalue would need to be in an appropriate
type (arguably the type of the piddle itself) but maybe just
the same as the new Core type for symmetry.

I haven't looked at this for a long time, but I seem to remember I had two
modes - one where a single badvalue was used for all piddles of that fata
type and one where the badvalue was per-piddle (so two float piddles could
have different badvalues), controlled by a build-time config. So either it
needs to be done in both places, one of these options removed, or my memory
checked as I don't know what I'm talking about.

Doug

[Craig DeForest]

That sounds good to me, Chris. My only concern, process-wise,
is that the change to the coercion is going to require monkeying
around with the very files I'm currently scrubbing. It might be
worthwhile to fix it in the core-cleanup branch now and then merge
the cleanup-in-progress to make a 2.008...

While going to a union would mean the end of the current shortcut
(of sequentially casting a test value to each type, and choosing the
first one that passes an equality test when cast back to double),
that shortcut is arguably pretty slow. Although it would be
a larger chunk of code, I think successive range checking would
probably run faster. (The code is generated in pdlcore.c.PL;
the code in question is pdl_whichdatatype).

Cheers,
Craig

Post by Chris Marshall
I would like to wrap up the 64bit index support for PDL-2.008
by fixing the longlong to double type coercion in the PDL Core.
This would entail changing the default any-type PDL value
from double to something else.
Similarly, the badvalue would need to be in an appropriate
type (arguably the type of the piddle itself) but maybe just
the same as the new Core type for symmetry.
It seems that a reasonable option might be a union type of
all the elemental PDL types plus pointers. Which types are
to be elemental could be refined later but we could start
with the current PDL types. The motivation for the pointer
is that we would use pass by reference to handle the most
general cases (conveniently, it also allows for all the perl
SV*, AV*,... as well).
For 2.008 there would be just enough "glue" to resolve the
lingering issues for longlong and 64bit PDL_Indx data. After
that release, I suggest we switch from the master branch to
working with the newly cleaned up core that Craig has been
working on.
Thoughts?
Chris
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters

[Chris Marshall]

I'll be trying in a branch and as minimally invasive as possible.

My hope is a monkey patch that can get correct 64bit index
support in December with a jump to development going forward
based on the cleaned-up core.

As is, the current broken longlong support means that
memory mapped file IO on 64bit size files is broken too.
That was one of the major goals for having 64bit indexing
in PDL.

--Chris


On Wed, Nov 27, 2013 at 11:41 AM, Craig DeForest

Post by Craig DeForest
That sounds good to me, Chris. My only concern, process-wise,
is that the change to the coercion is going to require monkeying
around with the very files I'm currently scrubbing. It might be
worthwhile to fix it in the core-cleanup branch now and then merge
the cleanup-in-progress to make a 2.008...
While going to a union would mean the end of the current shortcut
(of sequentially casting a test value to each type, and choosing the
first one that passes an equality test when cast back to double),
that shortcut is arguably pretty slow. Although it would be
a larger chunk of code, I think successive range checking would
probably run faster. (The code is generated in pdlcore.c.PL;
the code in question is pdl_whichdatatype).
Cheers,
Craig

Post by Chris Marshall
I would like to wrap up the 64bit index support for PDL-2.008
by fixing the longlong to double type coercion in the PDL Core.
This would entail changing the default any-type PDL value
from double to something else.
Similarly, the badvalue would need to be in an appropriate
type (arguably the type of the piddle itself) but maybe just
the same as the new Core type for symmetry.
It seems that a reasonable option might be a union type of
all the elemental PDL types plus pointers. Which types are
to be elemental could be refined later but we could start
with the current PDL types. The motivation for the pointer
is that we would use pass by reference to handle the most
general cases (conveniently, it also allows for all the perl
SV*, AV*,... as well).
For 2.008 there would be just enough "glue" to resolve the
lingering issues for longlong and 64bit PDL_Indx data. After
that release, I suggest we switch from the master branch to
working with the newly cleaned up core that Craig has been
working on.
Thoughts?
Chris
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters