Class: Sass::Script::Lexer

Inherits:
Object
  • Object
show all
Includes:
Sass::SCSS::RX
Defined in:
/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb

Overview

The lexical analyzer for SassScript. It takes a raw string and converts it to individual tokens that are easier to parse.

Direct Known Subclasses

CssLexer

Defined Under Namespace

Classes: Token

Constant Summary

Instance Method Summary (collapse)

Methods included from Sass::SCSS::RX

escape_ident

Constructor Details

- (Lexer) initialize(str, line, offset, options)

Returns a new instance of Lexer

Parameters:

  • str (String, StringScanner)

    The source text to lex

  • line (Integer)

    The 1-based line on which the SassScript appears. Used for error reporting and sourcemap building

  • offset (Integer)

    The 1-based character (not byte) offset in the line in the source. Used for error reporting and sourcemap building

  • options ({Symbol => Object})

    An options hash; see the Sass options documentation



153
154
155
156
157
158
159
160
161
162
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 153

def initialize(str, line, offset, options)
  @scanner = str.is_a?(StringScanner) ? str : Sass::Util::MultibyteStringScanner.new(str)
  @line = line
  @offset = offset
  @options = options
  @interpolation_stack = []
  @prev = nil
  @tok = nil
  @next_tok = nil
end

Instance Method Details

- (Boolean) after_interpolation?

Returns Whether or not the last token lexed was :end_interpolation.

Returns:

  • (Boolean)

    Whether or not the last token lexed was :end_interpolation.



217
218
219
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 217

def after_interpolation?
  @prev && @prev.type == :end_interpolation
end

- (String) char(pos = @scanner.pos)

Returns the given character.

Returns:

  • (String)


189
190
191
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 189

def char(pos = @scanner.pos)
  @scanner.string[pos, 1]
end

- (Boolean) done?

Returns Whether or not there's more source text to lex.

Returns:

  • (Boolean)

    Whether or not there's more source text to lex.



210
211
212
213
214
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 210

def done?
  return if @next_tok
  whitespace unless after_interpolation? && !@interpolation_stack.empty?
  @scanner.eos? && @tok.nil?
end

- expected!(name)

Raise an error to the effect that name was expected in the input stream and wasn't found.

This calls #unpeek! to rewind the scanner to immediately after the last returned token.

Parameters:

  • name (String)

    The name of the entity that was expected but not found

Raises:



229
230
231
232
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 229

def expected!(name)
  unpeek!
  Sass::SCSS::Parser.expected(@scanner, name, @line)
end

- (Integer) line

The line number of the lexer's current position.

Returns:

  • (Integer)


29
30
31
32
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 29

def line
  return @line unless @tok
  @tok.source_range.start_pos.line
end

- (Token) next

Moves the lexer forward one token.

Returns:

  • (Token)

    The token that was moved past



167
168
169
170
171
172
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 167

def next
  @tok ||= read_token
  @tok, tok = nil, @tok
  @prev = tok
  tok
end

- (Integer) offset

The number of bytes into the current line of the lexer's current position (1-based).

Returns:

  • (Integer)


38
39
40
41
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 38

def offset
  return @offset unless @tok
  @tok.source_range.start_pos.offset
end

- (Token) peek

Returns the next token without moving the lexer forward.

Returns:

  • (Token)

    The next token



196
197
198
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 196

def peek
  @tok ||= read_token
end

- (String) str { ... }

Records all non-comment text the lexer consumes within the block and returns it as a string.

Yields:

  • A block in which text is recorded

Returns:

  • (String)


239
240
241
242
243
244
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 239

def str
  old_pos = @tok ? @tok.pos : @scanner.pos
  yield
  new_pos = @tok ? @tok.pos : @scanner.pos
  @scanner.string[old_pos...new_pos]
end

- unpeek!

Rewinds the underlying StringScanner to before the token returned by #peek.



202
203
204
205
206
207
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 202

def unpeek!
  return unless @tok
  @scanner.pos = @tok.pos
  @line = @tok.source_range.start_pos.line
  @offset = @tok.source_range.start_pos.offset
end

- (Boolean) whitespace?(tok = @tok)

Returns whether or not there's whitespace before the next token.

Returns:

  • (Boolean)


177
178
179
180
181
182
183
184
# File '/home/nweiz/code/sass-site/.sass/lib/sass/script/lexer.rb', line 177

def whitespace?(tok = @tok)
  if tok
    @scanner.string[0...tok.pos] =~ /\s\Z/
  else
    @scanner.string[@scanner.pos, 1] =~ /^\s/ ||
      @scanner.string[@scanner.pos - 1, 1] =~ /\s\Z/
  end
end