jfun.parsec
Interface Tokenizer

All Superinterfaces:

java.io.Serializable

All Known Implementing Classes:

ConstTokenizer, TokenString

public interface Tokenizer
extends java.io.Serializable

A tokenizer is used by a scanner object to interpret the output of a Scanner.

Author:

Ben Yu 2004-11-7

Method Summary

java.lang.Object

toToken(java.lang.CharSequence cs, int from, int len) interprets the matched input range to a token object.

Method Detail

toToken

java.lang.Object toToken(java.lang.CharSequence cs,
                         int from,
                         int len)

interprets the matched input range to a token object.

Parameters:

cs - - the input

from - - the starting point

len - - the length of the range

Returns:

- the token object.