[IronPython] Is it possible to get the DLR's AST when it fails to parse?

Dino Viehland dinov at microsoft.com
Tue Dec 1 16:14:57 PST 2009


Tokenization would save some work but he could also just use the IronPython
parser.  It's unfortunately a small pain to create (it's overly tied to the
DLR APIs right now) but it's:

return Parser.CreateParser(
                new CompilerContext(sourceUnit, new PythonCompilerOptions(), ErrorSink.Null),
                new PythonOptions()
                );

And to get that SourceUnit you can do HostingHelpers.GetSourceUnit on the ScriptSource.

The resulting parsed expression will include ErrorExpression's where 
we encounter invalid code.  You may also find that things like class and
function definitions can be missing their names if you are parsing 
incomplete code like "def ".  Then you'll be working on the IronPython AST
instead of the DLR AST (but the two will be the same likely in 
IronPython 2.6.1 - the ASTs are now reducible DLR ASTs).


> -----Original Message-----
> From: users-bounces at lists.ironpython.com [mailto:users-
> bounces at lists.ironpython.com] On Behalf Of Michael Foord
> Sent: Tuesday, December 01, 2009 4:07 PM
> To: Discussion of IronPython
> Subject: Re: [IronPython] Is it possible to get the DLR's AST when it fails to
> parse?
> 
> No time for a longer reply, but you probably want to tokenize rather
> than parse.
> 
> Michael
> 
> Lee Culver wrote:
> >
> > I have been playing with an implementation of a simple GUI console for
> > IronPython (similar to IDLE, without it's text-editor IDE). I've
> > mostly brought it up to rough feature parity with IDLE, but I'd like
> > to add a rudimentary itellisense such that when you type something on
> > the order of "foo.", the interpreter can pop up a menu containing the
> > contents of foo (assuming it's something already defined in the local
> > scope). However, I'd really rather not get in the business of parsing
> > Python code. (That's what the DLR is for!)
> >
> > Is there any way I can use DLR to give me the AST up to the point at
> > which it failed to parse so that I can walk that and see if we are in
> > a valid place to look up the fields of an object? I looked at using
> > the ErrorListener class with ScriptSource.Compile, but it only gives
> > me "unexpected <eof> found" at the point where the "." is...which I
> > already knew.
> >
> > Basically, whenever the user presses the "." key in the console what I
> > would like to do is roughly this code:
> >
> > ScriptEngine engine = Python.CreateEngine();
> >
> > ScriptScope scope = engine.CreateScope();
> >
> > ScriptSource source = engine.CreateScriptSourceFromString("if foo.",
> > Microsoft.Scripting.SourceCodeKind.InteractiveCode);
> >
> > source.Compile(); // <-- This *should* fail. I want the AST when it does.
> >
> > Primarily, I'm trying to find chains of identifiers to walk so that I
> > can display the Nth identifier's fields:
> >
> > identifier1 . identifier2 . identifier3 . ...
> >
> > I need to distinguish this from things like this, because there's
> > nothing you can reasonably do to have intellisense in this case:
> >
> > Function(args) . identifier1 . identifier2
> >
> > (Yes, I realize that walking this chain can execute code and have side
> > effects, I'm willing to live with that for my purposes.) Is there any
> > mechanism through which I can have the DLR parse this for me and give
> > me the resulting AST that I can walk so I can see if we can pop up an
> > intellisense-like view?
> >
> > Can anyone recommend a better way to do this outside of simply writing
> > my own mini-parser?
> >
> > Thanks,
> >
> > -Lee Culver
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Users mailing list
> > Users at lists.ironpython.com
> > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> >
> 
> 
> --
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
> 
> 
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


More information about the Users mailing list