PyPy gets a new compiler
Today, I merged the parser-compiler branch, which I have been working on over the summer. It contained a total rewrite of both PyPy's Python parser and AST compiler. PyPy's old parser was (in)famous internally for being complicated and slow (with many algorithmic complexities greater than O(n)). The new parser is a simple as I could make it LL(1) parser like CPython (though it doesn't share the hacks of CPython's parser).
The new compiler is based on the Abstract Syntax Trees (AST) that CPython 2.5 introduced instead of PyPy's old AST based on the compiler package's. This means that Python code running on PyPy will be able to use the same _ast interface as CPython. PyPy's _ast implementation supports AST features that CPython 2.6 added, including compiling modified AST to bytecode and executing it. In this rewrite, some more obscure compiler features were added, too. For example, jumps in bytecode can now be greater than 65535 bytes! (That's like an if statement with 7000 lines of code in the body.)
While the PyPy translation toolchain still has many obscure details and hacks, this merge completes the process of making the actual Python interpreter very clean. Hopefully, this will make adding new features much easier and make PyPy less frustrating to maintain as well as providing application level code with an improved AST interface!
Comments
Nice, keep up the good work!
Thank you.. Keep it up.
Very nice. Thanks for all of your work!
Hi, the Gothenburg sprint news are very interesting.
What are your thoughts about a release roadmap?. Do you intend to release a pypy 1.2 with improved compatibility and speed but no JIT, and later include the JIT (version 1.5, maybe?)?.
I think publishing some kind of roadmap would be useful, as a project suffers when its release cycles are BOTH long and unpredictable.
Also, starting from the next stable release, it would be great to publish some kind of benchmarks page to keep track of performance across different versions (cpython 2.6 vs pypy 1.1 vs pypy 1.2 vs pypy with JIT).
Now that I think of it, do you need some kind of help with the website?. I think starting with the next pypy's release, the project will get a lot more visibility and a nicer and better structured website would be a definite plus. If you feel it would be a useful task I could help there.
Hey.
Both, the benchmarks (that would also include say jython) and a nice website for people who actually want to use it would be a very nice addon. We definitely would appreciate some help with it.
If you have any ideas feel free to continue discussion on pypy-dev.
Cheers,
fijal
Hi Maciej, as you suggested, I have subscribed to the pypy-dev mailing list and have started the discussion.
Cheers,
Miquel
Hey Miguel.
I fail to see your post.
Cheers,
fijal
it got rejected. I have written to pypy-dev-owner to see where the problem is.
Cheers
@tobami
you should subscribe to the list first.
We get far too much spam to accept
posts from non-members.
Cheers,
fijal
@Maciej,
well, I subscribed first, that is the problem. I now get sent the pypy-dev mailing list, but my post got rejected anyway. And pypy-owner hasn't answered yet.
What can I do?
@tobami
you did something wrong. pypy-dev
is not a moderated list (from
members, that is). Can you leave your mail, so we can no longer spam here? Mine is fijal at merlinux.eu