lexer

Documentation

Index

• Variables
• type Config
• type Lexer
  • func New(input string, config Config) *Lexer
  • func (l *Lexer) NextToken() token.Token

Variables

var DefaultConfig = Config{
	AllowComments:    true,
	StringDelimiters: []byte{'"'},
}

DefaultLexerConfig defines default settings for the lexer

Types

type Config

type Config struct {
	AllowComments    bool
	StringDelimiters []byte
}

LexerConfig provides configuration options for the lexer

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer performs lexical analysis on the input source code

func New

func New(input string, config Config) *Lexer

New creates a new Lexer instance with the provided configuration

func (*Lexer) NextToken

func (l *Lexer) NextToken() token.Token

NextToken retrieves the next token from the input