org.openoffice.da.comp.w2lcommon.tex.tokenizer
Class Token

java.lang.Object
  org.openoffice.da.comp.w2lcommon.tex.tokenizer.Token

All Implemented Interfaces:

java.lang.Cloneable

public class Token
extends java.lang.Object
implements java.lang.Cloneable

This class represent a token in TeX

Constructor Summary

Token()
Construct a new Token, initialized as a TokenTYPE.ENDINPUT-token

Method Summary

protected void	addChar(char c) Append a character to the character content of this token
protected void	clearChars() Delete the character content of this token
char	getChar() Get the first character in this token
java.lang.String	getString() Get the character content of this token as a string
TokenType	getType() Get the TokenType of this token
boolean	is(char c, TokenType type) Test wether this token is a character token of the given type (that is, a single character with a token type that is neither COMMAND_SEQUENCE nor ENDINPUT)
boolean	isCS(java.lang.String sName) Test wether this token is a COMMAND_SEQUENCE token with a given name
protected void	set(char c, TokenType type) Set this token as a character token with a specific TokenType
protected void	setChar(char c) Set the character content of this token to a single character (the type of the token is not changed)
protected void	setType(TokenType type) Set the type of this token to a specific TokenType (the character content is not changed)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Token

public Token()

Construct a new Token, initialized as a TokenTYPE.ENDINPUT-token

Method Detail

setType

protected void setType(TokenType type)

Set the type of this token to a specific TokenType (the character content is not changed)

Parameters:

type - the new TokenType

setChar

protected void setChar(char c)

Set the character content of this token to a single character (the type of the token is not changed)

Parameters:

c - the character

set

protected void set(char c,
                   TokenType type)

Set this token as a character token with a specific TokenType

Parameters:

c - the character

type - the TokenType to use

clearChars

protected void clearChars()

Delete the character content of this token

addChar

protected void addChar(char c)

Append a character to the character content of this token

Parameters:

c - the character to be appended

is

public boolean is(char c,
                  TokenType type)

Test wether this token is a character token of the given type (that is, a single character with a token type that is neither COMMAND_SEQUENCE nor ENDINPUT)

Parameters:

c - the character to test

type - the TokenType to test

Returns:

true if the test was successful

isCS

public boolean isCS(java.lang.String sName)

Test wether this token is a COMMAND_SEQUENCE token with a given name

Parameters:

sName - the name of the command sequence

Returns:

true if the test was successful

getType

public TokenType getType()

Get the TokenType of this token

Returns:

the type

getChar

public char getChar()

Get the first character in this token

Returns:

the character or U+FFFF is no characters exist

getString

public java.lang.String getString()

Get the character content of this token as a string

Returns:

the character content

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object