QYc@sdZddlZddlmZddlmZddlmZddlmZddlmZde fd YZ ed e fd YZ dS( sTop down operator precedence parser. This is an implementation of Vaughan R. Pratt's "Top Down Operator Precedence" parser. (http://dl.acm.org/citation.cfm?doid=512927.512931). These are some additional resources that help explain the general idea behind a Pratt parser: * http://effbot.org/zone/simple-top-down-parsing.htm * http://javascript.crockford.com/tdop/tdop.html A few notes on the implementation. * All the nud/led tokens are on the Parser class itself, and are dispatched using getattr(). This keeps all the parsing logic contained to a single class. * We use two passes through the data. One to create a list of token, then one pass through the tokens to create the AST. While the lexer actually yields tokens, we convert it to a list so we can easily implement two tokens of lookahead. A previous implementation used a fixed circular buffer, but it was significantly slower. Also, the average jmespath expression typically does not have a large amount of token so this is not an issue. And interestingly enough, creating a token list first is actually faster than consuming from the token iterator one token at a time. iN(tlexer(twith_repr_method(tast(t exceptions(tvisitortParsercBseZidd6dd6dd6dd6dd6dd6dd6dd6dd 6dd 6dd 6dd 6d d6dd6dd6dd6dd6dd6dd6dd6dd6dd6dd6dd6d d!6d"d#6d$d%6d&d'6d(d)6Zd*ZiZd+Zdd,Zd-Zd.Zd/Z dd0Z d1Z d2Z d3Z d4Zd5Zd6Zd7Zd8Zd9Zd:Zd;Zd<Zd=Zd>Zd?Zd@ZdAZdBZdCZdDZdEZdFZ dGZ!dHZ"dIZ#dJZ$dKZ%dLZ&dMZ'dNZ(dOZ)dPZ*dQZ+dRZ,dSZ-dTZ.d_dUZ0dVZ1dWZ2dXZ3dYZ4dZZ5d[Z6d\Z7d]Z8e9d^Z:RS(`iteoftunquoted_identifiertquoted_identifiertliteraltrbrackettrparentcommatrbracetnumbertcurrenttexpreftcolonitpipeitoritanditeqtgttlttgtetltetnei tflattenitstaritfilteri(tdoti-tnoti2tlbracei7tlbracketi<tlpareni icCs/d|_dg||_||_d|_dS(Ni(tNonet tokenizert_tokenst _buffer_sizet_index(tselft lookahead((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt__init__Ns  cCsg|jj|}|dk r"|S|j|}||j|s _token_led_%s(RDt_advancetgetattrt_error_nud_tokenRCt BINDING_POWERR#t_error_led_token(R(R;t left_tokent nud_functiontleftt current_tokentledt error_token((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRBvs        cCstj|dS(NR=(RR (R(ttoken((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_literalscCstj|dS(NR=(Rtfield(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_unquoted_identifierscCs[tj|d}|jdkrW|jd}tjd|d|ddn|S(NR=R"iR>s1Quoted identifier not allowed for function names.(RRURCRDRR9(R(RSRURG((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_quoted_identifiers cCsStj}|jdkr-tj}n|j|jd}tj||S(NR R(RtidentityRCt_parse_projection_rhsRKtvalue_projection(R(RSROtright((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_stars  cCs|jtjS(N(t_token_led_filterRRX(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_filterscCs |jS(N(t_parse_multi_select_hash(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lbracescCs|j}|jd|S(NR (RBt_match(R(RSR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lparens  cCs;tjtj}|j|jd}tj||S(NR(RRRXRYRKt projection(R(RSROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_flattenscCs#|j|jd}tj|S(NR(RBRKRtnot_expression(R(RStexpr((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_notscCs|jdkr4|j}|jtj|S|jdkr|jddkr|j|j|j|jd}tj tj|S|j SdS(NRRRiR (snumberR( RCt_parse_index_expressiont_project_if_sliceRRXt _lookaheadRHRYRKRct_parse_multi_select_list(R(RSR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_lbrackets   cCso|jddks*|jddkr4|jStj|jdd}|j|jd|SdS(NiRiR=R (Rjt_parse_slice_expressionRtindexRDRHRa(R(tnode((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRhs   cCsdddg}d}|j}x|dk r|dkr|dkr|d7}|dkr{|j|jddn|jnI|dkr|jdd||<|jn|j|jdd|j}q$W|jdtj|S( NiR iRis syntax errorRR=(R#RCt_raise_parse_error_for_tokenRDRHRaRtslice(R(tpartsRnRP((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRms$        cCs tjS(N(Rt current_node(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_currentscCs#|j|jd}tj|S(NR(RBRKRR(R(RSR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_nud_exprefscCs|jdksc|j|jd}|ddkrM|dj||Stj||gSn0|j|j|jd}tj||SdS(NRRR>t subexpressiontchildren( RCt_parse_dot_rhsRKtappendRRvRHRYRZ(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_dots cCs&|j|jd}tj||S(NR(RBRKRR(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_pipescCs&|j|jd}tj||S(NR(RBRKRt or_expression(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_or scCs&|j|jd}tj||S(NR(RBRKRtand_expression(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_and scCs|ddkrN|jd}tj|d|d|dd|dn|d}g}xQ|jdks|j}|jdkr|jdn|j|qaW|jdtj||}|S( NR>RUiR<R=sInvalid function name '%s'R R ( RDRR9RCRBRaRyRtfunction_expression(R(ROtprev_ttnametargsR1t function_node((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lparens   cCsf|jd}|jd|jdkr=tj}n|j|jd}tj|||S(NiR RR(RBRaRCRRXRYRKtfilter_projection(R(ROt conditionR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR]%s  cCs|j|dS(NR(t_parse_comparator(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_eq/scCs|j|dS(NR(R(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_ne2scCs|j|dS(NR(R(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_gt5scCs|j|dS(NR(R(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_gte8scCs|j|dS(NR(R(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt _token_led_lt;scCs|j|dS(NR(R(R(RO((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lte>scCs5tj|}|j|jd}tj||S(NR(RRRYRKRc(R(ROR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_flattenAscCs|jd}|dd krc|j}|ddkrP|dj||S|j||Sn@|jd|jd|j|jd}tj||SdS( NiR>RRtindex_expressionRwRR (snumberscolon( RDRhRyRiRaRYRKRRc(R(RORSR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_token_led_lbracketGs   cCsMtj||g}|ddkrEtj||j|jdS|SdS(NR>RqR(RRRcRYRK(R(ROR[t index_expr((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRiZs cCs)|j|j|}tj|||S(N(RBRKRt comparator(R(RORR[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRcscCsig}xFtrN|j}|j||jdkr>Pq |jdq W|jdtj|S(NR R (tTrueRBRyRCRaRtmulti_select_list(R(t expressionsR1((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRkgs    cCsg}xtr|jd}|jdddg|d}|jd|jd}tjd|d|}|j||jd kr|jd q |jd kr |jd Pq q Wtj d |S( Nit token_typesRRR=Rtkey_nameRoR R tnodes( RRDt_match_multiple_tokensRaRBRt key_val_pairRyRCtmulti_select_dict(R(tpairst key_tokenRR=Ro((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR_ss        cCs|j|j|jkr+tj}n|jdkrO|j|}nn|jdkrs|j|}nJ|jdkr|jd|j|}n|j|j dd|S(NR!RRis syntax error( RKRCt_PROJECTION_STOPRRXRBRaRxRpRD(R(R;R[((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRYs cCs|j}|d kr%|j|S|dkrH|jd|jS|dkrk|jd|jS|jd}ddddg}d||df}|j||dS( NRRRR!R isExpecting: %s, got: %sR>(squoted_identifiersunquoted_identifiersstar(RCRBRaRkR_RDRp(R(R;R)RGtallowedtmsg((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRxs          cCsH|ddkr4tj|d|d|dn|j|ddS(NR>RR<R=s invalid token(RR7Rp(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRJscCs|j|ddS(Ns invalid token(Rp(R(RS((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRLscCs<|j|kr|jn|j||jddS(Ni(RCRHt_raise_parse_error_maybe_eofRD(R(t token_type((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRas cCs<|j|kr.|j||jdn|jdS(Ni(RCRRDRH(R(R((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRscCs|jd7_dS(Ni(R'(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRHscCs|j|jdS(NR>(R%R'(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRCscCs|j|j|dS(NR>(R%R'(R(R((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRjscCs|j|j|S(N(R%R'(R(R((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRDscCs:|d}|d}|d}tj||||dS(NR<R=R>(RR9(R(RStreasont lex_positiont actual_valuet actual_type((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRps     cCsn|d}|d}|d}|dkrBtj|||nd||f}tj||||dS(NR<R=R>RsExpecting: %s, got: %s(RR7R9(R(t expected_typeRSRRRtmessage((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyRs     cCsAx:tj|jjt|jdD]}|j|=q)WdS(Ni(trandomtsampleR+tkeystintR/(R(tkey((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR0s/cCs|jjdS(s'Clear the expression compilation cache.N(R+tclear(tcls((s3/usr/lib/python2.7/site-packages/jmespath/parser.pytpurgesN(;t__name__t __module__RKRR+R/R*R4R-R5RBRTRVRWR\R^R`RbRdRgRlRhRmRtRuRzR{R}RRR]RRRRRRRRRiRRkR_RYRxRJRLR#RaRRHRCRjRDRpRR0t classmethodR(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR%s                                          REcBs/eZdZddZdZdZRS(cCs||_||_dS(N(R1RF(R(R1RF((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyR*s cCs(tj|}|j|j|}|S(N(RtTreeInterpretertvisitRF(R(R=toptionst interpretertresult((s3/usr/lib/python2.7/site-packages/jmespath/parser.pytsearchscCs"tj}|j|j}|S(sfRender the parsed AST as a dot file. Note that this is marked as an internal method because the AST is an implementation detail and is subject to change. This method can be used to help troubleshoot or for development purposes, but is not considered part of the public supported API. Use at your own risk. (RtGraphvizVisitorRRF(R(trenderertcontents((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt_render_dot_files cCs t|jS(N(treprRF(R(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyt__repr__sN(RRR*R#RRR(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyREs   ( t__doc__RtjmespathRtjmespath.compatRRRRtobjectRRE(((s3/usr/lib/python2.7/site-packages/jmespath/parser.pyts