[Pdl-porters] PDL3 needs combine-scan operations

[Chris Marshall]

Continuing sequence about feature ideas for PDL3...

My recent threading experience ended up using the
indadd routine to accumulate the final results. In parallel
programming and various array/functional programming
environments, there are existing concepts and routines
that address this problem.

PDL2 has rudimentary data combining capabilities, actually
only indadd comes to mind. indadd is fine for 1D work but
it get tricky to get right for higher dimensional calculations
and it is awkward to read and not general.

I propose that we add support for combine-scan and
combine-prefix operations to PDL. The idea is essentially
to combine indexing with various combining (binary
associative) operations. For example, the default .= is
essentially copy and you are left with the last value that
was assigned in the thread loops (i.e., undefined if the
indices are not 1-to-1 in the output).

indadd corresponds to a 1D version where the combining
operation is addition.

The combine-prefix operations are basically the same
but they result in a vector of values that correspond to
the partial sums (comibines of the operation) along the
vector dimension. HPF has some discussion for a fortran
implementation, see sections 7.4.4 and 7.4.5:

http://dacnet.rice.edu/versions/hpf2/hpf-v20.pdf

In addition to this basis for PDL3 functionality, I would
like to see a clean way to incorporate the capability
in a simple to follow and use way. For example, some
languages has special notation that indicates the
combining operation. Here is a made-up example with
sort-of perl/PDL syntax:

$dest ++= $src->indexND($inds);

where the fake operator ++= means that the += operation
is being performed using the + combiner for elements.

Comments,
Chris

[David Mertens]

I think I like this idea, but I don't think I quite understand what you're
driving at. Where would this be used apart from, say, indexmult? Could you
provide a few toy examples?

David

Post by Chris Marshall
Continuing sequence about feature ideas for PDL3...
My recent threading experience ended up using the
indadd routine to accumulate the final results. In parallel
programming and various array/functional programming
environments, there are existing concepts and routines
that address this problem.
PDL2 has rudimentary data combining capabilities, actually
only indadd comes to mind. indadd is fine for 1D work but
it get tricky to get right for higher dimensional calculations
and it is awkward to read and not general.
I propose that we add support for combine-scan and
combine-prefix operations to PDL. The idea is essentially
to combine indexing with various combining (binary
associative) operations. For example, the default .= is
essentially copy and you are left with the last value that
was assigned in the thread loops (i.e., undefined if the
indices are not 1-to-1 in the output).
indadd corresponds to a 1D version where the combining
operation is addition.
The combine-prefix operations are basically the same
but they result in a vector of values that correspond to
the partial sums (comibines of the operation) along the
vector dimension. HPF has some discussion for a fortran
http://dacnet.rice.edu/versions/hpf2/hpf-v20.pdf
In addition to this basis for PDL3 functionality, I would
like to see a clean way to incorporate the capability
in a simple to follow and use way. For example, some
languages has special notation that indicates the
combining operation. Here is a made-up example with
$dest ++= $src->indexND($inds);
where the fake operator ++= means that the += operation
is being performed using the + combiner for elements.
Comments,
Chris
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters

[Craig DeForest]

There are a lot of similar things that I currently use one-off PP operators for. A good example is photon target practice -- the same one that is called out as not working right in the range() docs. Given a list of photon "hit" locations, make a 2-D histogram image. (Sort of like indadd, but N-dimensional).

I think I like this idea, but I don't think I quite understand what you're driving at. Where would this be used apart from, say, indexmult? Could you provide a few toy examples?
David
Continuing sequence about feature ideas for PDL3...
My recent threading experience ended up using the
indadd routine to accumulate the final results. In parallel
programming and various array/functional programming
environments, there are existing concepts and routines
that address this problem.
PDL2 has rudimentary data combining capabilities, actually
only indadd comes to mind. indadd is fine for 1D work but
it get tricky to get right for higher dimensional calculations
and it is awkward to read and not general.
I propose that we add support for combine-scan and
combine-prefix operations to PDL. The idea is essentially
to combine indexing with various combining (binary
associative) operations. For example, the default .= is
essentially copy and you are left with the last value that
was assigned in the thread loops (i.e., undefined if the
indices are not 1-to-1 in the output).
indadd corresponds to a 1D version where the combining
operation is addition.
The combine-prefix operations are basically the same
but they result in a vector of values that correspond to
the partial sums (comibines of the operation) along the
vector dimension. HPF has some discussion for a fortran
http://dacnet.rice.edu/versions/hpf2/hpf-v20.pdf
In addition to this basis for PDL3 functionality, I would
like to see a clean way to incorporate the capability
in a simple to follow and use way. For example, some
languages has special notation that indicates the
combining operation. Here is a made-up example with
$dest ++= $src->indexND($inds);
where the fake operator ++= means that the += operation
is being performed using the + combiner for elements.
Comments,
Chris
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters

[Chris Marshall]

sum_scan can be used to count or get statistics (mean...)
min_scan and max_scan can give bounds generation
prod_scan can be used for index calculation, dimension loops
sum_prefix can be used for distributed implementation of loops

Sorry this is high level.
Chris

Post by David Mertens
I think I like this idea, but I don't think I quite understand what you're
driving at. Where would this be used apart from, say, indexmult? Could you
provide a few toy examples?
David

Post by Chris Marshall
Continuing sequence about feature ideas for PDL3...
My recent threading experience ended up using the
indadd routine to accumulate the final results. In parallel
programming and various array/functional programming
environments, there are existing concepts and routines
that address this problem.
PDL2 has rudimentary data combining capabilities, actually
only indadd comes to mind. indadd is fine for 1D work but
it get tricky to get right for higher dimensional calculations
and it is awkward to read and not general.
I propose that we add support for combine-scan and
combine-prefix operations to PDL. The idea is essentially
to combine indexing with various combining (binary
associative) operations. For example, the default .= is
essentially copy and you are left with the last value that
was assigned in the thread loops (i.e., undefined if the
indices are not 1-to-1 in the output).
indadd corresponds to a 1D version where the combining
operation is addition.
The combine-prefix operations are basically the same
but they result in a vector of values that correspond to
the partial sums (comibines of the operation) along the
vector dimension. HPF has some discussion for a fortran
http://dacnet.rice.edu/versions/hpf2/hpf-v20.pdf
In addition to this basis for PDL3 functionality, I would
like to see a clean way to incorporate the capability
in a simple to follow and use way. For example, some
languages has special notation that indicates the
combining operation. Here is a made-up example with
$dest ++= $src->indexND($inds);
where the fake operator ++= means that the += operation
is being performed using the + combiner for elements.
Comments,
Chris
_______________________________________________
PDL-porters mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan