third_party.pylibs.pylint.src/pylint/testutils/tokenize_str.py

# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
# For details: https://github.com/PyCQA/pylint/blob/main/LICENSE

import tokenize
from io import StringIO


def _tokenize_str(code):
    return list(tokenize.generate_tokens(StringIO(code).readline))
Burst what remain of utils into files 2020-11-29 10:17:56 +00:00			`# Licensed under the GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html`
Fix copyright links (#4647) * Fix link in license header * Update link to astroid bump_changelog 2021-07-01 10:47:58 +00:00			`# For details: https://github.com/PyCQA/pylint/blob/main/LICENSE`
Burst what remain of utils into files 2020-11-29 10:17:56 +00:00
			`import tokenize`
			`from io import StringIO`


			`def _tokenize_str(code):`
			`return list(tokenize.generate_tokens(StringIO(code).readline))`