[IronPython] [Fwd: Re: [pyconuk] Parallel Processing and Python]

Sat Oct 4 03:22:24 PDT 2008

Out of a discussion on multithreading on the PyCon UK mailing list - it 
turns out that Kamaelia (a concurrency library) 'mainly' works on 
IronPython 2 now - which is great news. I believe Michael Sparks will be 
filing at least one bug report - but some issues that used to prevent it 
working have been fixed.

-------- Original Message --------
Subject: 	Re: [pyconuk] Parallel Processing and Python
Date: 	Fri, 3 Oct 2008 16:59:38 +0100
From: 	Michael Sparks <sparks.m at gmail.com>
Organization: 	Just little old me
To: 	Michael Foord <fuzzyman at voidspace.org.uk>
CC: 	pyconuk at python.org, kamaelia at googlegroups.com, Discussion of 
IronPython <users at lists.ironpython.com>
References: 	<48D00408.50705 at simplistix.co.uk> 
<200810031620.42992.sparks.m at gmail.com> <48E639E1.4040602 at voidspace.org.uk>

> Amongst everything you posted below I could only *see* problems with
> using setuptools and distutils 

Correct.

> - neither of which I particularly expect to work unmodified in IronPython
> for the *forseeable* future.  

Ahhh, I didn't realise that. That means fundamentally you can view it as 
driver error in that case.  If there's a preferred packaging scheme for 
IronPython, I'll see if we can support that as well.

I wasn't aware of the various points, that explains a lot. Well in which case, 
more works than can be expected :)

> As far as I could *see* actually *using* Kamaelia worked fine (other
> than the echoer server client disconnection issue you mentioned) - am I
> reading that right? If so it is a massive step forward and fantastic news.

Yes, it is a massive step forward and it is *fantastic* news.

Whilst the examples I posted are relatively simple, I know which areas that 
are getting exercised by this - like putting select in a different thread, 
communications using Queue.Queue, things like multiple yield statements in 
generators etc.

This does mean that all the examples I've posted will fairly trivially be 
multicore :-)

Since posting the last message, I've also tested TCPClient, which used to 
exercise a particular bug in IronPython because of this structure:

[snip]
                  yield 2
               raise Finality
            except Exception, x:
               result = sock.shutdown(2) ; yield 3
               raise x  # XXXX If X is not finality.. [snip]
         except Exception, x:
            sock.close() ;  yield 4,x # XXXX If X is not finality.. [snip]
            raise x
      except Finality:
         yield 5
      except socket.error, e:
         # We now do the flipside of setupCSA, whether we had an error or not
         # A safe error relates to a disconnected server, and unsafe error is
[snip]

In case that looks odd, it's due to the issue that try:...finally:... is not 
legal inside a generator in python pre-2.5. This particular structure used to 
cause problems for IronPython, and I'm glad to see that's been fixed:

C:\Program Files\IronPython 2.0 Beta 5>ipy.exe
IronPython 2.0 Beta (2.0.0.5000) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.
>>> import Axon
>>> import Kamaelia
>>> from Kamaelia.Chassis.Pipeline import Pipeline
>>> from Kamaelia.Util.Console import *
>>> from Kamaelia.Internet.TCPClient import TCPClient
>>> Pipeline(
...     ConsoleReader(),
...     TCPClient("132.185.142.2", 80),
...     ConsoleEchoer(),
... ).run()

Then the output of that run: (GET .. etc is typed by me)

>>> GET / HTTP/1.0
>>>
>>> HTTP/1.1 200 OK
Date: Fri, 03 Oct 2008 14:19:40 GMT
Server: Apache/2.0.49 (Linux/SuSE)
Last-Modified: Wed, 21 May 2008 21:14:05 GMT
ETag: "32663b2-6f-139fa140"
Accept-Ranges: bytes
Content-Length: 111
Connection: close
Content-Type: text/html; charset=ISO-8859-1

<html>
<body bgcolor="#ffffff">

<p> Today's fish is trout ala creme, please enjoy your meal.

</body>
</html>
Traceback (most recent call last):
  File "", line unknown, in <module>
  File "Snippets.scripting", line unknown, in run
  File "Snippets.scripting", line unknown, in runThreads
  File "Microsoft.Scripting.Core", line unknown, in MoveNext
  File "Snippets.scripting", line unknown, in main
ValueError: list.index(item): item not in list

> Could you post details of the disconnection issue?

I had wanted to check whether it was a Kamaelia bug. However, it looks like it 
is an IronPython issue. However, running the above example shows up the same 
bug in IronPython - where as it functions correctly on CPython.

:-)

That said, I really think that is a massive jump forward for Kamaelia on 
IronPython and think it's fantastic news :)

Regards,

Michael.

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/