kR\ @s{dZddlmZddlmZddlmZmZmZddl m Z Gddde Z dd d Z dS) z SQL Lexer)tokens) SQL_REGEX) bytes_type text_type file_types)consumec@s+eZdZdZedddZdS)Lexerz?Lexer Empty class. Leaving for backwards-compatibility Nccsut|tr|j}nt|tr0n|t|tr|rW|j|}qy|jd}Wqtk r|jd}YqXntdjt |t |}x|D]\}}xt D]\}}|||}|sqnFt|t j r||jfVn t|r?||jVnt||j|dPqWt j|fVqWdS)a Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. Also preprocess the text, i.e. expand tabs and strip it if wanted and applies registered filters. Split ``text`` into (tokentype, text) pairs. ``stack`` is the inital stack (default: ``['root']``) zutf-8zunicode-escapez+Expected text or file-like object, got {!r}N) isinstancerreadrrdecodeUnicodeDecodeError TypeErrorformattype enumeraterrZ _TokenTypegroupcallablerendError)textencodingiterableposcharZrexmatchactionmr6/var/www/dbchiro/venv/build/sqlparse/sqlparse/lexer.py get_tokenss4    zLexer.get_tokens)__name__ __module__ __qualname____doc__ staticmethodrrrrrrs rNcCstj||S)zTokenize sql. Tokenize *sql* using the :class:`Lexer` and return a 2-tuple stream of ``(token type, value)`` items. )rr)ZsqlrrrrtokenizeKsr%)r#ZsqlparserZsqlparse.keywordsrZsqlparse.compatrrrZsqlparse.utilsrobjectrr%rrrrs 6