token_get_all
(PHP 4 >= 4.2.0, PHP 5, PHP 7)
token_get_all — Split given source into PHP tokens
Description
$source
[, int $flags = 0
] ) : array
token_get_all() parses the given source
string into PHP language tokens using the Zend engine's lexical scanner.
For a list of parser tokens, see List of Parser Tokens, or use token_name() to translate a token value into its string representation.
Parameters
-
source -
The PHP source to parse.
-
flags -
Valid flags:
-
TOKEN_PARSE- Recognises the ability to use reserved words in specific contexts.
-
Return Values
An array of token identifiers. Each individual token identifier is either a single character (i.e.: ;, ., >, !, etc...), or a three element array containing the token index in element 0, the string content of the original token in element 1 and the line number in element 2.
Changelog
| Version | Description |
|---|---|
| 7.0.0 |
Added the optional flags parameter along with
the TOKEN_PARSE flag.
|
| 5.2.2 | Line numbers are returned in element 2 |
Examples
Example #1 token_get_all() example
<?php
$tokens = token_get_all('<?php echo; ?>');
foreach ($tokens as $token) {
if (is_array($token)) {
echo "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL;
}
}
?>
The above example will output something similar to:
Line 1: T_OPEN_TAG ('<?php ')
Line 1: T_ECHO ('echo')
Line 1: T_WHITESPACE (' ')
Line 1: T_CLOSE_TAG ('?>')
Example #2 token_get_all() incorrect usage example
<?php
$tokens = token_get_all('/* comment */');
foreach ($tokens as $token) {
if (is_array($token)) {
echo "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL;
}
}
?>
The above example will output something similar to:
Line 1: T_INLINE_HTML ('/* comment */')
T_INLINE_HTML rather than the expected
T_COMMENT. This is because no open tag was used in the
code provided. This would be equivalent to putting a comment outside of the
PHP tags in a normal file.
Example #3 token_get_all() on a class using a reserved word example
<?php
$source = <<<'code'
<?php
class A
{
const PUBLIC = 1;
}
code;
$tokens = token_get_all($source, TOKEN_PARSE);
foreach ($tokens as $token) {
if (is_array($token)) {
echo token_name($token[0]) , PHP_EOL;
}
}
?>
The above example will output something similar to:
T_OPEN_TAG T_WHITESPACE T_CLASS T_WHITESPACE T_STRING T_CONST T_WHITESPACE T_STRING T_LNUMBER
TOKEN_PARSE flag, the penultimate
token (T_STRING) would have been
T_PUBLIC.
English translation
You have asked to visit this site in English. For now, only the interface is translated, but not all the content yet.If you want to help me in translations, your contribution is welcome. All you need to do is register on the site, and send me a message asking me to add you to the group of translators, which will give you the opportunity to translate the pages you want. A link at the bottom of each translated page indicates that you are the translator, and has a link to your profile.
Thank you in advance.
Document created the 30/01/2003, last modified the 26/10/2018
Source of the printed document:https://www.gaudry.be/en/php-rf-token-get-all.html
The infobrol is a personal site whose content is my sole responsibility. The text is available under CreativeCommons license (BY-NC-SA). More info on the terms of use and the author.
References
These references and links indicate documents consulted during the writing of this page, or which may provide additional information, but the authors of these sources can not be held responsible for the content of this page.
The author This site is solely responsible for the way in which the various concepts, and the freedoms that are taken with the reference works, are presented here. Remember that you must cross multiple source information to reduce the risk of errors.
