Token
Represents a token.
Tags
Table of Contents
- NONE = 0
- End of text or error.
- PUNCTUATION = 2
- Punctuation token.
- UNKNOWN = 4
- Character not used in any of the supported natural languages.
- WHITESPACE = 3
- Whitespace token.
- WORD = 1
- Word token.
- $text : string
- Token text.
- $type : int
- Token type.
Constants
NONE
End of text or error.
public
mixed
NONE
= ""
PUNCTUATION
Punctuation token.
public
mixed
PUNCTUATION
= 2
UNKNOWN
Character not used in any of the supported natural languages.
public
mixed
UNKNOWN
= 4
WHITESPACE
Whitespace token.
public
mixed
WHITESPACE
= 3
WORD
Word token.
public
mixed
WORD
= 1
Properties
$text
Token text.
public
string
$text
$type
Token type.
public
int
$type