l i n u x - u s e r s - g r o u p - o f - d a v i s
L U G O D
 
Next Meeting:
January 6: Social gathering
Next Installfest:
TBD
Latest News:
Nov. 18: Club officer elections
Page last updated:
2011 Jun 17 20:19

The following is an archive of a post made to our 'vox mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox] Who thinks Java is cool?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox] Who thinks Java is cool?



On Thu, Jun 16, 2011 at 11:59:50PM -0700, Bill Broadley wrote:
> On 06/15/2011 11:41 PM, Norm Matloff wrote:

> > It's generally believed in the parallel
> > processing community that the shared-memory paradigm makes for clearer
> > code than does message-passing, and I personally agree.
 
> That's an interesting assertion, possibly better discussed in person.

Well, there's fact and there's taste.

I think it's fair to say that it is a fact that shared-memory code is
simpler, i.e. takes up fewer lines of source code.

But is simpler clearer?  That is absolutely a matter of taste.

> The HPC world of parallel processing seems to be largely invested in MPI
> which seems orders of magnitude more popular than shared memory both in
> HPC clusters and in HPC codes.  

That was true until two or three years ago.  But then GPU became big.

> Sure OpenMP works, but that's only for toy scaling.  If you want
> to scale to say a factor of 100, 1000 or 10,000 on a code that's not
> embarrassingly parallel then you use message passing.  

In general, one can do more tweaking in a message-passing environment.
This is especially true if the alternative is a cache-coherent
shared-memory setting.  That picture changes radically with GPU.  If
one's criterion is how many different applications can get excellent
performance/price ratio, GPU would probably win hands down.

Of course, one does even better in a multi-GPU setting, linking the GPUs
using message-passing, and currently this is the typical mode in the
"world class" supercomputers.  (Take that with a grain of salt, as the
applications used are rather narrow.)

I'm really surprised that you say one can get speedups of multiple
orders of magnitude on non-embarrassingly parallel problems.  I've been
trying to think up examples of this, or find examples that others have
implemented, for quite a while now, and I haven't found any.  If you
know of examples, PLEASE LET ME KNOW.  The all-caps format here is not
to shout at you or do challenge you, but rather because it really
matters.  Among other things, I'm writing a book on R programming, and I
have a chapter on parallel R.  Things get a little more complicated in
R, but the basic issue remains:  Can one achieve speedups of that
magnitude on non-embarrassingly parallel problems in large-scale
distributed systems?

> I've had a few experiences with MPI over shared memory (SGI and Mpich)
> and both worked rather poorly.  Granted that's not relevant to getting
> useful things done with shared memory.  As counter intuitive as it was,
> it was faster to send a message to a pci-e attached card for local
> messages than it was to send a message using shared memory.

Yes, I've had a similar experience.  For those doing message-passing, I
don't recommend using shared memory as the underlying communications
medium.

> Go has pointers, but not pointer arithmetic (unless I'm
> misremembering my languages).

Then does it not have shared arrays?

Norm

_______________________________________________
vox mailing list
vox@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox



LinkedIn
LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
facebook
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
O'Reilly and Associates
For numerous book donations.