Compilers (CS362 2002F)

Using Lexical-Analyzer Generators

Summary: In today's lab, you will explore the standard Unix lexical analyzer generator, lex.

Collaboration: Feel free to work on this lab in pairs or trios.

Turning It In: Save your answers in a plain text file and submit it using the ECA.

Grading: I expect that you will gain more from doing this lab than from me grading this lab. I will simply scan through your answers to see if you had any particularly valuable insights.

Background: lex (or flex, in its GNU implementation) is one of the standard lexical analyzer generators. You can use lex to build lexers and also to build programs that react in other ways to regular expressions they encounter.

You can get more information on flex from the man page. Please don't print the man page; it's over sixty pages long. I'll keep a copy at the front of the room.

Structure of a Lex File:

c-code-prefixed-by-tabs-and-named-regular-expressions
%%
pattern1	c-code-to-execute-upon-matching-pattern
pattern2	c-code-to-execute-upon-matching-pattern
...
patternn	c-code-to-execute-upon-matching-pattern
%%
more-c-code

Sample Lex File:

%%
" "		{ }
\n              { printf("Howdy!\n"); }
begin           { printf("Keyword-Begin\n"); }
[a-z][a-z0-9_]* { printf("Look, an identifier: '%s'\n",yytext); }
%%
int yywrap() { return 1; }

The yywrap function gets called by the lexer whenever it hits the end of file. If yywrap returns a nonzero value, the lexer stops.

Using Lex Files:

Suppose we saved this as example.yy (yes, .yy is the normal suffix for lex files). To build an executable from it, you would do the following:

% flex example.yy
	# Generates file lex.yy.c
% gcc -lfl lex.yy.c
	# Generates file a.out
% a.out < file

Problems:

1. Figure out what the example file does with things not described in the regular expressions.

2. Use flex to separate input into

3. Look at the lex.yy.c and see if you can figure out where and how the finite automaton is coded. You might want to try variants of the lex program to see what difference it makes.

 

History

Monday, 16 September 2002

  • Created, based on Andrew Kensler's notes from the last time I gave this lab. (That version of the lab was not written up.)

Tuesday, 17 September 2002

  • Added a few more notes.

 

Disclaimer: I usually create these pages on the fly, which means that I rarely proofread them and they may contain bad grammar and incorrect details. It also means that I tend to update them regularly (see the history for more details). Feel free to contact me with any suggestions for changes.

This document was generated by Siteweaver on Tue Dec 10 08:53:26 2002.
The source to the document was last modified on Tue Sep 17 08:49:55 2002.
This document may be found at http://www.cs.grinnell.edu/~rebelsky/Courses/CS362/2002F/Labs/lab.03.html.

You may wish to validate this document's HTML ; Valid CSS! ; Check with Bobby

Glimmer Labs: The Grinnell Laboratory for Interactive Multimedia Experimentation & Research
glimmer@grinnell.edu