[IronPython] Performance of IronPython 2 Beta 4 and IronPython 1

Dino Viehland dinov at exchange.microsoft.com
Thu Aug 14 14:44:04 PDT 2008


IronPython displays it when it starts up on the first line: "... on .NET 2.0.50727.3031"

-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Michael Foord
Sent: Thursday, August 14, 2008 2:34 PM
To: Discussion of IronPython
Subject: Re: [IronPython] Performance of IronPython 2 Beta 4 and IronPython 1

Curt Hagenlocher wrote:
> Can you tell us about the environment you're running these tests in?
> Most interesting are the specific OS and the version of mscorwks.dll.
>

It's Win XP SP 3. What's the easiest way of telling you the specific
version of mscorwks.dll? I'll get you more details from work tomorrow.

Michael

> On Thu, Aug 14, 2008 at 9:41 AM, Michael Foord
> <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>> wrote:
>
>     Just for fun I also compared with CPython. The results are
>     interesting, I'll turn it into a blog post of course...
>
>
>     Results in milliseconds with a granularity of about 15ms and so an
>     accuracy of +/- ~60ms.
>     All testing with 10 000 000 operations unless otherwise stated.
>
>     The version of Python I compared against was Python 2.4.
>
>     Empty loop (overhead):
>       IP1: 422
>       IP2: 438
>       Py: 3578
>
>      Create instance newstyle:
>       IP1: 20360
>       IP2:  1109
>       Py:   4063
>
>      Create instance oldstyle:
>       IP1: 3766
>       IP2: 3359
>       Py:  4797
>
>      Function call:
>       IP1: 937
>       IP2: 906
>       Py: 3313
>
>      Create function: 25% slower
>       IP1: 2828
>       IP2: 3640
>       Py:  2766
>
>      Define newstyle (1 000 000):
>       IP1: 42047
>       IP2: 20484
>       Py:  23921
>
>      Define oldstyle (1 000 000): 33% slower
>       IP1: 1781
>       IP2: 2671
>       Py:  2108
>
>
>     Comparing (== and !=):
>       IP1: 278597
>       IP2: 117662
>       Py:   62423
>      Sets:
>       IP1: 37095
>       IP2: 30860
>       Py:   8047
>
>
>     Lists (10 000): 50% slower
>       IP1: 10422
>       IP2: 16109
>       Py:   6094
>
>
>     Recursion (10 000):
>       IP1: 1125
>       IP2: 1000
>       Py:  3609
>
>      Sets2 (100 000): 500% slower
>       IP1:  4984
>       IP2: 30547
>       Py:   1203
>
>     func_with_args:
>       IP1: 6312
>       IP2: 5906
>       Py: 11250
>
>     method_with_args:
>       IP1: 20594
>       IP2: 11813
>       Py:  14875
>      method_with_kwargs:
>       IP1: 27953
>       IP2: 11187
>       Py:  20032
>      import: 15% slower
>       IP1: 28469
>       IP2: 32000
>       Py:  25782
>
>     global: 20% slower
>       IP1: 1047
>       IP2: 1203
>       Py:  4141
>
>     Exceptions (100000): 40% slower
>       IP1: 4703
>       IP2: 6125
>       Py:   266
>      Engine execution: 8000% slower!!
>       IP1: 1600
>       IP2: 115002
>
>
>     Michael Foord wrote:
>
>         Hello all,
>
>         I've ported Resolver One to run on IronPython 2 Beta 4 to
>         check for any potential problems (we will only do a *proper*
>         port once IP 2 is out of beta).
>
>         The basic porting was straightforward and several bugs have
>         been fixed since IP 2 B3 - many thanks to the IronPython team.
>
>         The good news is that Resolver One is only 30-50% slower than
>         Resolver One on IronPython 1! (It was 300 - 400% slower on top
>         of IP 2 B3.) Resolver One is fairly heavily optimised around
>         the performance hotspots of IronPython 1, so we expect to have
>         to do a fair bit of profiling and refactoring to readjust to
>         the performance profile of IP 2.
>
>         Having said that, there are a few oddities (and the areas that
>         slow down vary tremendously depending on which spreadsheet we
>         use to benchmark it - making it fairly difficult to track down
>         the hotspots).
>
>         We have one particular phase of spreadsheet calculation that
>         takes 0.4seconds on IP1 and around 6 seconds on IP2, so I have
>         been doing some micro-benchmarking to try and identify the
>         hotspot. I've certainly found part of the problem.
>
>         For those that are interested I've attached the very basic
>         microbenchmarks I've been using. The nice thing is that in
>         *general* IP2 does outperform IP1.
>
>         The results that stand out in the other direction are:
>
>         Using sets with custom classes (that define '__eq__', '__ne__'
>         and '__hash__') seems to be 6 times slower in IronPython 2.
>
>         Adding lists together is about 50% slower.
>
>         Defining functions seems to be 25% slower and defining old
>         style classes about 33% slower. (Creating instances of new
>         style classes is massively faster though - thanks!)
>
>         The code I used to test sets (sets2.py) is as follows:
>
>         from System import DateTime
>
>         class Thing(object):
>           def __init__(self, val):
>               self.val = val
>             def __eq__(self, other):
>               return self.val == other.val
>
>           def __neq__(self):
>               return not self.__eq__(other)
>                 def __hash__(self):
>               return hash(self.val)
>                    def test(s):
>           a = set()
>           for i in xrange(100000):
>               a.add(Thing(i))
>               a.add(Thing(i+1))
>               Thing(i) in a
>               Thing(i+2) in a
>           return (DateTime.Now -s).TotalMilliseconds
>          s = DateTime.Now
>         print test(s)
>
>
>         Interestingly the time taken is exactly the same if I remove
>         the definition of '__hash__'.
>
>         The full set of results below:
>
>         Results in milliseconds with a granularity of about 15ms and
>         so an accuracy of +/- ~60ms.
>         All testing with 10 000 000 operations unless otherwise stated.
>
>         Empty loop (overhead):
>           IP1: 421.9
>           IP2: 438
>          Create instance newstyle:
>           IP1: 20360
>           IP2: 1109
>          Create instance oldstyle:
>           IP1: 3766
>           IP2: 3359
>          Function call:
>           IP1: 937
>           IP2: 906
>          Create function: 25% slower
>           IP1: 2828
>           IP2: 3640
>          Define newstyle (1 000 000):
>           IP1: 42047
>           IP2: 20484
>          Define oldstyle (1 000 000): 33% slower
>           IP1: 1781
>           IP2: 2671
>
>         Comparing (== and !=):
>           IP1: 278597
>           IP2: 117662
>          Sets (with numbers):
>           IP1: 37095
>           IP2: 30860
>
>         Lists (10 000): 50% slower
>           IP1: 10422
>           IP2: 16109
>
>         Recursion (10 000):
>           IP1: 1125
>           IP2: 1000
>          Sets2 (100 000): 600% slower
>           IP1: 4984
>           IP2: 30547
>
>
>         I'll be doing more as the 600% slow down for sets and the 50%
>         slow down for lists accounts for some of the dependency
>         analysis problem but not all of it.
>
>         Many Thanks
>
>         Michael Foord
>         --
>         http://www.resolversystems.com <http://www.resolversystems.com/>
>         http://www.ironpythoninaction.com
>         <http://www.ironpythoninaction.com/>
>
>
>
>         ------------------------------------------------------------------------
>
>         _______________________________________________
>         Users mailing list
>         Users at lists.ironpython.com <mailto:Users at lists.ironpython.com>
>         http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>     --
>     http://www.ironpythoninaction.com/
>     http://www.voidspace.org.uk/
>     http://www.trypython.org/
>     http://www.ironpython.info/
>     http://www.resolverhacks.net/
>     http://www.theotherdelia.co.uk/
>
>
>     _______________________________________________
>     Users mailing list
>     Users at lists.ironpython.com <mailto:Users at lists.ironpython.com>
>     http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/

_______________________________________________
Users mailing list
Users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

Sample disclaimer text


More information about the Users mailing list