The Concepts of Bison
This chapter introduces many of the basic concepts without which the details of Bison will not make sense. If you do not already know how to use Bison or Yacc, we suggest you start by reading this chapter carefully.Languages and Context-Free Grammars
In order for Bison to parse a language, it must be described by a context-free grammar. This means that you specify one or more syntactic groupings and give rules for constructing them from their parts. For example, in the C language, one kind of grouping is called an `expression'. One rule for making an expression might be, "An expression can be made of a minus sign and another expression". Another would be, "An expression can be an integer". As you can see, rules are often recursive, but there must be at least one rule which leads out of the recursion.The most common formal system for presenting such rules for humans to read is Backus-Naur Form or "BNF", which was developed in order to specify the language Algol 60. Any grammar expressed in BNF is a context-free grammar. The input to Bison is essentially machine-readable BNF.
Not all context-free languages can be handled by Bison, only those that are LALR(1). In brief, this means that it must be possible to tell how to parse any portion of an input string with just a single token of look-ahead. Strictly speaking, that is a description of an LR(1) grammar, and LALR(1) involves additional restrictions that are hard to explain simply; but it is rare in actual practice to find an LR(1) grammar that fails to be LALR(1). See section Mysterious Reduce/Reduce Conflicts, for more information on this.
In the formal grammatical rules for a language, each kind of syntactic unit or grouping is named by a symbol. Those which are built by grouping smaller constructs according to grammatical rules are called nonterminal symbols; those which can't be subdivided are called terminal symbols or token types. We call a piece of input corresponding to a single terminal symbol a token, and a piece corresponding to a single nonterminal symbol a grouping.
We can use the C language as an example of what symbols, terminal and nonterminal, mean. The tokens of C are identifiers, constants (numeric and string), and the various keywords, arithmetic operators and punctuation marks. So the terminal symbols of a grammar for C include `identifier', `number', `string', plus one symbol for each keyword, operator or punctuation mark: `if', `return', `const', `static', `int', `char', `plus-sign', `open-brace', `close-brace', `comma' and many more. (These tokens can be subdivided into characters, but that is a matter of lexicography, not grammar.)
Here is a simple C function subdivided into tokens:
int /* keyword `int' */
square (x) /* identifier, open-paren, */
/* identifier, close-paren */
int x; /* keyword `int', identifier, semicolon */
{ /* open-brace */
return x * x; /* keyword `return', identifier, */
/* asterisk, identifier, semicolon */
} /* close-brace */
The syntactic groupings of C include the expression, the statement, the declaration, and the function definition. These are represented in the grammar of C by nonterminal symbols `expression', `statement', `declaration' and `function definition'. The full grammar uses dozens of additional language constructs, each with its own nonterminal symbol, in order to express the meanings of these four. The example above is a function definition; it contains one declaration, and one statement. In the statement, each `x' is an expression and so is `x * x'.Each nonterminal symbol must have grammatical rules showing how it is made out of simpler constructs. For example, one kind of C statement is the
return statement; this would be described with a grammar rule which reads informally as follows:A `statement' can be made of a `return' keyword, an `expression' and a `semicolon'.There would be many other rules for `statement', one for each kind of statement in C.
One nonterminal symbol must be distinguished as the special one which defines a complete utterance in the language. It is called the start symbol. In a compiler, this means a complete input program. In the C language, the nonterminal symbol `sequence of definitions and declarations' plays this role.
For example, `1 + 2' is a valid C expression--a valid part of a C program--but it is not valid as an entire C program. In the context-free grammar of C, this follows from the fact that `expression' is not the start symbol.
The Bison parser reads a sequence of tokens as its input, and groups the tokens using the grammar rules. If the input is valid, the end result is that the entire token sequence reduces to a single grouping whose symbol is the grammar's start symbol. If we use a grammar for C, the entire input must be a `sequence of definitions and declarations'. If not, the parser reports a syntax error.
From Formal Rules to Bison Input
A formal grammar is a mathematical construct. To define the language for Bison, you must write a file expressing the grammar in Bison syntax: a Bison grammar file. See section Bison Grammar Files.A nonterminal symbol in the formal grammar is represented in Bison input as an identifier, like an identifier in C. By convention, it should be in lower case, such as
expr, stmt or declaration.The Bison representation for a terminal symbol is also called a token type. Token types as well can be represented as C-like identifiers. By convention, these identifiers should be upper case to distinguish them from nonterminals: for example,
INTEGER, IDENTIFIER, IF or RETURN. A terminal symbol that stands for a particular keyword in the language should be named after that keyword converted to upper case. The terminal symbolerror is reserved for error recovery. See section Symbols, Terminal and Nonterminal.A terminal symbol can also be represented as a character literal, just like a C character constant. You should do this whenever a token is just a single character (parenthesis, plus-sign, etc.): use that same character in a literal as the terminal symbol for that token.
The grammar rules also have an expression in Bison syntax. For example, here is the Bison rule for a C
return statement. The semicolon in quotes is a literal character token, representing part of the C syntax for the statement; the naked semicolon, and the colon, are Bison punctuation used in every rule.stmt: RETURN expr ';'
;
See section Syntax of Grammar Rules.Semantic Values
A formal grammar selects tokens only by their classifications: for example, if a rule mentions the terminal symbol `integer constant', it means that any integer constant is grammatically valid in that position. The precise value of the constant is irrelevant to how to parse the input: if `x+4' is grammatical then `x+1' or `x+3989' is equally grammatical.But the precise value is very important for what the input means once it is parsed. A compiler is useless if it fails to distinguish between 4, 1 and 3989 as constants in the program! Therefore, each token in a Bison grammar has both a token type and a semantic value. See section Defining Language Semantics, for details.
The token type is a terminal symbol defined in the grammar, such as
INTEGER, IDENTIFIER or ','. It tells everything you need to know to decide where the token may validly appear and how to group it with other tokens. The grammar rules know nothing about tokens except their types.The semantic value has all the rest of the information about the meaning of the token, such as the value of an integer, or the name of an identifier. (A token such as
',' which is just punctuation doesn't need to have any semantic value.)For example, an input token might be classified as token type
INTEGER and have the semantic value 4. Another input token might have the same token type INTEGER but value 3989. When a grammar rule says thatINTEGER is allowed, either of these tokens is acceptable because each is an INTEGER. When the parser accepts the token, it keeps track of the token's semantic value.Each grouping can also have a semantic value as well as its nonterminal symbol. For example, in a calculator, an expression typically has a semantic value that is a number. In a compiler for a programming language, an expression typically has a semantic value that is a tree structure describing the meaning of the expression.
Semantic Actions
In order to be useful, a program must do more than parse input; it must also produce some output based on the input. In a Bison grammar, a grammar rule can have an action made up of C statements. Each time the parser recognizes a match for that rule, the action is executed. See section Actions. Most of the time, the purpose of an action is to compute the semantic value of the whole construct from the semantic values of its parts. For example, suppose we have a rule which says an expression can be the sum of two expressions. When the parser recognizes such a sum, each of the subexpressions has a semantic value which describes how it was built up. The action for this rule should create a similar sort of value for the newly recognized larger expression.For example, here is a rule that says an expression can be the sum of two subexpressions:
expr: expr '+' expr { $$ = $1 + $3; }
;
The action says how to produce the semantic value of the sum expression from the values of the two subexpressions.Bison Output: the Parser File
When you run Bison, you give it a Bison grammar file as input. The output is a C source file that parses the language described by the grammar. This file is called a Bison parser. Keep in mind that the Bison utility and the Bison parser are two distinct programs: the Bison utility is a program whose output is the Bison parser that becomes part of your program.The job of the Bison parser is to group tokens into groupings according to the grammar rules--for example, to build identifiers and operators into expressions. As it does this, it runs the actions for the grammar rules it uses.
The tokens come from a function called the lexical analyzer that you must supply in some fashion (such as by writing it in C). The Bison parser calls the lexical analyzer each time it wants a new token. It doesn't know what is "inside" the tokens (though their semantic values may reflect this). Typically the lexical analyzer makes the tokens by parsing characters of text, but Bison does not depend on this. See section The Lexical Analyzer Function
yylex.The Bison parser file is C code which defines a function named
yyparse which implements that grammar. This function does not make a complete C program: you must supply some additional functions. One is the lexical analyzer. Another is an error-reporting function which the parser calls to report an error. In addition, a complete C program must start with a function called main; you have to provide this, and arrange for it to call yyparseor the parser will never run. See section Parser C-Language Interface.Aside from the token type names and the symbols in the actions you write, all variable and function names used in the Bison parser file begin with `yy' or `YY'. This includes interface functions such as the lexical analyzer function
yylex, the error reporting function yyerror and the parser function yyparse itself. This also includes numerous identifiers used for internal purposes. Therefore, you should avoid using C identifiers starting with`yy' or `YY' in the Bison grammar file except for the ones defined in this manual.Stages in Using Bison
The actual language-design process using Bison, from grammar specification to a working compiler or interpreter, has these parts:- Formally specify the grammar in a form recognized by Bison (see section Bison Grammar Files). For each grammatical rule in the language, describe the action that is to be taken when an instance of that rule is recognized. The action is described by a sequence of C statements.
- Write a lexical analyzer to process input and pass tokens to the parser. The lexical analyzer may be written by hand in C (see section The Lexical Analyzer Function
yylex). It could also be produced using Lex, but the use of Lex is not discussed in this manual. - Write a controlling function that calls the Bison-produced parser.
- Write error-reporting routines.
- Run Bison on the grammar to produce the parser.
- Compile the code output by Bison, as well as any other source files.
- Link the object files to produce the finished product.
The Overall Layout of a Bison Grammar
The input file for the Bison utility is a Bison grammar file. The general form of a Bison grammar file is as follows:%{
C declarations
%}
Bison declarations
%%
Grammar rules
%%
Additional C code
The `%%', `%{' and `%}' are punctuation that appears in every Bison grammar file to separate the sections.The C declarations may define types and variables used in the actions. You can also use preprocessor commands to define macros used there, and use
#include to include header files that do any of these things.The Bison declarations declare the names of the terminal and nonterminal symbols, and may also describe operator precedence and the data types of semantic values of various symbols.
The grammar rules define how to construct each nonterminal symbol from its parts.
The additional C code can contain any C code you want to use. Often the definition of the lexical analyzer
yylex goes here, plus subroutines called by the actions in the grammar rules. In a simple program, all the rest of the program can go here.Bison Grammar Files
Bison takes as input a context-free grammar specification and produces a C-language function that recognizes correct instances of the grammar.The Bison grammar input file conventionally has a name ending in `.y'.
Outline of a Bison Grammar
A Bison grammar file has four main sections, shown here with the appropriate delimiters:%{
C declarations
%}
Bison declarations
%%
Grammar rules
%%
Additional C code
Comments enclosed in `/* ... */' may appear in any of the sections.The C Declarations Section
The C declarations section contains macro definitions and declarations of functions and variables that are used in the actions in the grammar rules. These are copied to the beginning of the parser file so that they precede the definition ofyyparse. You can use `#include' to get the declarations from a header file. If you don't need any C declarations, you may omit the `%{' and `%}' delimiters that bracket this section.The Bison Declarations Section
The Bison declarations section contains declarations that define terminal and nonterminal symbols, specify precedence, and so on. In some simple grammars you may not need any declarations. See section Bison Declarations.The Grammar Rules Section
The grammar rules section contains one or more Bison grammar rules, and nothing else. See section Syntax of Grammar Rules.There must always be at least one grammar rule, and the first `%%' (which precedes the grammar rules) may never be omitted even if it is the first thing in the file.
The Additional C Code Section
The additional C code section is copied verbatim to the end of the parser file, just as the C declarations section is copied to the beginning. This is the most convenient place to put anything that you want to have in the parser file but which need not come before the definition ofyyparse. For example, the definitions of yylex and yyerror often go here. See section Parser C-Language Interface.If the last section is empty, you may omit the `%%' that separates it from the grammar rules.
The Bison parser itself contains many static variables whose names start with `yy' and many macros whose names start with `YY'. It is a good idea to avoid using any such names (except those documented in this manual) in the additional C code section of the grammar file.
Symbols, Terminal and Nonterminal
Symbols in Bison grammars represent the grammatical classifications of the language.A terminal symbol (also known as a token type) represents a class of syntactically equivalent tokens. You use the symbol in grammar rules to mean that a token in that class is allowed. The symbol is represented in the Bison parser by a numeric code, and the
yylex function returns a token type code to indicate what kind of token has been read. You don't need to know what the code value is; you can use the symbol to stand for it.A nonterminal symbol stands for a class of syntactically equivalent groupings. The symbol name is used in writing grammar rules. By convention, it should be all lower case.
Symbol names can contain letters, digits (not at the beginning), underscores and periods. Periods make sense only in nonterminals.
There are two ways of writing terminal symbols in the grammar:
- A named token type is written with an identifier, like an identifier in C. By convention, it should be all upper case. Each such name must be defined with a Bison declaration such as
%token. See section Token Type Names. - A character token type (or literal token) is written in the grammar using the same syntax used in C for character constants; for example,
'+'is a character token type. A character token type doesn't need to be declared unless you need to specify its semantic value data type (see section Data Types of Semantic Values), associativity, or precedence (see section Operator Precedence).By convention, a character token type is used only to represent a token that consists of that particular character. Thus, the token type'+'is used to represent the character `+' as a token. Nothing enforces this convention, but if you depart from it, your program will confuse other readers.
All the usual escape sequences used in character literals in C can be used in Bison as well, but you must not use the null character as a character literal because its ASCII code, zero, is the codeyylexreturns for end-of-input (see section Calling Convention foryylex).
The value returned by
yylex is always one of the terminal symbols (or 0 for end-of-input). Whichever way you write the token type in the grammar rules, you write it the same way in the definition of yylex. The numeric code for a character token type is simply the ASCII code for the character, so yylex can use the identical character constant to generate the requisite code. Each named token type becomes a C macro in the parser file, soyylex can use the name to stand for the code. (This is why periods don't make sense in terminal symbols.) See section Calling Convention for yylex.If
yylex is defined in a separate file, you need to arrange for the token-type macro definitions to be available there. Use the `-d' option when you run Bison, so that it will write these macro definitions into a separate header file `name.tab.h' which you can include in the other source files that need it. See section Invoking Bison.The symbol
error is a terminal symbol reserved for error recovery (see section Error Recovery); you shouldn't use it for any other purpose. In particular, yylex should never return this value.Syntax of Grammar Rules
A Bison grammar rule has the following general form:result: components...
;
where result is the nonterminal symbol that this rule describes and components are various terminal and nonterminal symbols that are put together by this rule (see section Symbols, Terminal and Nonterminal).For example,
exp: exp '+' exp
;
says that two groupings of type exp, with a `+' token in between, can be combined into a larger grouping of type exp.Whitespace in rules is significant only to separate symbols. You can add extra whitespace as you wish.
Scattered among the components can be actions that determine the semantics of the rule. An action looks like this:
{C statements}
Usually there is only one action and it follows the components. See section Actions.Multiple rules for the same result can be written separately or can be joined with the vertical-bar character `|' as follows:
result: rule1-components...
| rule2-components...
...
;
They are still considered distinct rules even when joined in this way.If components in a rule is empty, it means that result can match the empty string. For example, here is how to define a comma-separated sequence of zero or more
exp groupings:expseq: /* empty */
| expseq1
;
expseq1: exp
| expseq1 ',' exp
;
It is customary to write a comment `/* empty */' in each rule with no components.Recursive Rules
A rule is called recursive when its result nonterminal appears also on its right hand side. Nearly all Bison grammars need to use recursion, because that is the only way to define a sequence of any number of somethings. Consider this recursive definition of a comma-separated sequence of one or more expressions:expseq1: exp
| expseq1 ',' exp
;
Since the recursive use of expseq1 is the leftmost symbol in the right hand side, we call this left recursion. By contrast, here the same construct is defined using right recursion:expseq1: exp
| exp ',' expseq1
;
Any kind of sequence can be defined using either left recursion or right recursion, but you should always use left recursion, because it can parse a sequence of any number of elements with bounded stack space. Right recursion uses up space on the Bison stack in proportion to the number of elements in the sequence, because all the elements must be shifted onto the stack before the rule can be applied even once. See section The Bison Parser Algorithm, for further explanation of this.Indirect or mutual recursion occurs when the result of the rule does not appear directly on its right hand side, but does appear in rules for other nonterminals which do appear on its right hand side.
For example:
expr: primary
| primary '+' primary
;
primary: constant
| '(' expr ')'
;
defines two mutually-recursive nonterminals, since each refers to the other.Defining Language Semantics
The grammar rules for a language determine only the syntax. The semantics are determined by the semantic values associated with various tokens and groupings, and by the actions taken when various groupings are recognized.For example, the calculator calculates properly because the value associated with each expression is the proper number; it adds properly because the action for the grouping `x + y' is to add the numbers associated withx and y.
Data Types of Semantic Values
In a simple program it may be sufficient to use the same data type for the semantic values of all language constructs. This was true in the RPN and infix calculator examples (see section Reverse Polish Notation Calculator).Bison's default is to use type
int for all semantic values. To specify some other type, define YYSTYPE as a macro, like this:#define YYSTYPE doubleThis macro definition must go in the C declarations section of the grammar file (see section Outline of a Bison Grammar).
More Than One Value Type
In most programs, you will need different data types for different kinds of tokens and groupings. For example, a numeric constant may need typeint or long, while a string constant needs type char *, and an identifier might need a pointer to an entry in the symbol table.To use more than one data type for semantic values in one parser, Bison requires you to do two things:
- Specify the entire collection of possible data types, with the
%unionBison declaration (see section The Collection of Value Types). - Choose one of those types for each symbol (terminal or nonterminal) for which semantic values are used. This is done for tokens with the
%tokenBison declaration (see section Token Type Names) and for groupings with the%typeBison declaration (see section Nonterminal Symbols).
Actions
An action accompanies a syntactic rule and contains C code to be executed each time an instance of that rule is recognized. The task of most actions is to compute a semantic value for the grouping built by the rule from the semantic values associated with tokens or smaller groupings.An action consists of C statements surrounded by braces, much like a compound statement in C. It can be placed at any position in the rule; it is executed at that position. Most rules have just one action at the end of the rule, following all the components. Actions in the middle of a rule are tricky and used only for special purposes (see section Actions in Mid-Rule).
The C code in an action can refer to the semantic values of the components matched by the rule with the construct
$n, which stands for the value of the nth component. The semantic value for the grouping being constructed is $$. (Bison translates both of these constructs into array element references when it copies the actions into the parser file.)Here is a typical example:
exp: ...
| exp '+' exp
{ $$ = $1 + $3; }
This rule constructs an exp from two smaller exp groupings connected by a plus-sign token. In the action, $1 and $3 refer to the semantic values of the two component exp groupings, which are the first and third symbols on the right hand side of the rule. The sum is stored into $$ so that it becomes the semantic value of the addition-expression just recognized by the rule. If there were a useful semantic value associated with the `+' token, it could be referred to as $2.If you don't specify an action for a rule, Bison supplies a default:
$$ = $1. Thus, the value of the first symbol in the rule becomes the value of the whole rule. Of course, the default rule is valid only if the two data types match. There is no meaningful default action for an empty rule; every empty rule must have an explicit action unless the rule's value does not matter.$n with n zero or negative is allowed for reference to tokens and groupings on the stack before those that match the current rule. This is a very risky practice, and to use it reliably you must be certain of the context in which the rule is applied. Here is a case in which you can use this reliably:foo: expr bar '+' expr { ... }
| expr bar '-' expr { ... }
;
bar: /* empty */
{ previous_expr = $0; }
;
As long as bar is used only in the fashion shown here, $0 always refers to the expr which precedes bar in the definition of foo.Data Types of Values in Actions
If you have chosen a single data type for semantic values, the$$ and $n constructs always have that data type.If you have used
%union to specify a variety of data types, then you must declare a choice among these types for each terminal or nonterminal symbol that can have a semantic value. Then each time you use $$ or $n, its data type is determined by which symbol it refers to in the rule. In this example,exp: ...
| exp '+' exp
{ $$ = $1 + $3; }
$1 and $3 refer to instances of exp, so they all have the data type declared for the nonterminal symbol exp. If $2 were used, it would have the data type declared for the terminal symbol '+', whatever that might be.Alternatively, you can specify the data type when you refer to the value, by inserting `<type>' after the `$' at the beginning of the reference. For example, if you have defined types as shown here:
%union {
int itype;
double dtype;
}
then you can write $<itype>1 to refer to the first subunit of the rule as an integer, or $<dtype>1 to refer to it as a double.Actions in Mid-Rule
Occasionally it is useful to put an action in the middle of a rule. These actions are written just like usual end-of-rule actions, but they are executed before the parser even recognizes the following components.A mid-rule action may refer to the components preceding it using
$n, but it may not refer to subsequent components because it is run before they are parsed.The mid-rule action itself counts as one of the components of the rule. This makes a difference when there is another action later in the same rule (and usually there is another at the end): you have to count the actions along with the symbols when working out which number n to use in
$n.The mid-rule action can also have a semantic value. The action can set its value with an assignment to
$$, and actions later in the rule can refer to the value using $n. Since there is no symbol to name the action, there is no way to declare a data type for the value in advance, so you must use the `$<...>' construct to specify a data type each time you refer to this value.There is no way to set the value of the entire rule with a mid-rule action, because assignments to
$$ do not have that effect. The only way to set the value for the entire rule is with an ordinary action at the end of the rule.Here is an example from a hypothetical compiler, handling a
let statement that looks like `let (variable) statement' and serves to create a variable named variable temporarily for the duration of statement. To parse this construct, we must put variable into the symbol table while statement is parsed, then remove it afterward. Here is how it is done:stmt: LET '(' var ')'
{ $<context>$ = push_context ();
declare_variable ($3); }
stmt { $$ = $6;
pop_context ($<context>5); }
As soon as `let (variable)' has been recognized, the first action is run. It saves a copy of the current semantic context (the list of accessible variables) as its semantic value, using alternative context in the data-type union. Then it calls declare_variable to add the new variable to that list. Once the first action is finished, the embedded statement stmt can be parsed. Note that the mid-rule action is component number 5, so the`stmt' is component number 6.After the embedded statement is parsed, its semantic value becomes the value of the entire
let-statement. Then the semantic value from the earlier action is used to restore the prior list of variables. This removes the temporary let-variable from the list so that it won't appear to exist while the rest of the program is parsed.Taking action before a rule is completely recognized often leads to conflicts since the parser must commit to a parse in order to execute the action. For example, the following two rules, without mid-rule actions, can coexist in a working parser because the parser can shift the open-brace token and look at what follows before deciding whether there is a declaration or not:
compound: '{' declarations statements '}'
| '{' statements '}'
;
But when we add a mid-rule action as follows, the rules become nonfunctional:compound: { prepare_for_local_variables (); }
'{' declarations statements '}'
| '{' statements '}'
;
Now the parser is forced to decide whether to run the mid-rule action when it has read no farther than the open-brace. In other words, it must commit to using one rule or the other, without sufficient information to do it correctly. (The open-brace token is what is called the look-ahead token at this time, since the parser is still deciding what to do about it. See section Look-Ahead Tokens.)You might think that you could correct the problem by putting identical actions into the two rules, like this:
compound: { prepare_for_local_variables (); }
'{' declarations statements '}'
| { prepare_for_local_variables (); }
'{' statements '}'
;
But this does not help, because Bison does not realize that the two actions are identical. (Bison never tries to understand the C code in an action.)If the grammar is such that a declaration can be distinguished from a statement by the first token (which is true in C), then one solution which does work is to put the action after the open-brace, like this:
compound: '{' { prepare_for_local_variables (); }
declarations statements '}'
| '{' statements '}'
;
Now the first token of the following declaration or statement, which would in any case tell Bison which rule to use, can still do so.Another solution is to bury the action inside a nonterminal symbol which serves as a subroutine:
subroutine: /* empty */
{ prepare_for_local_variables (); }
;
compound: subroutine
'{' declarations statements '}'
| subroutine
'{' statements '}'
;
Now Bison can execute the action in the rule for subroutine without deciding which rule for compound it will eventually use. Note that the action is now at the end of its rule. Any mid-rule action can be converted to an end-of-rule action in this way, and this is what Bison actually does to implement mid-rule actions.Bison Declarations
The Bison declarations section of a Bison grammar defines the symbols used in formulating the grammar and the data types of semantic values. See section Symbols, Terminal and Nonterminal.All token type names (but not single-character literal tokens such as
'+' and '*') must be declared. Nonterminal symbols must be declared if you need to specify which data type to use for the semantic value (see sectionMore Than One Value Type).The first rule in the file also specifies the start symbol, by default. If you want some other symbol to be the start symbol, you must declare it explicitly (see section Languages and Context-Free Grammars).
Token Type Names
The basic way to declare a token type name (terminal symbol) is as follows:%token nameBison will convert this into a
#define directive in the parser, so that the function yylex (if it is in this file) can use the name name to stand for this token type's code.Alternatively, you can use
%left, %right, or %nonassoc instead of %token, if you wish to specify precedence. See section Operator Precedence.You can explicitly specify the numeric code for a token type by appending an integer value in the field immediately following the token name:
%token NUM 300It is generally best, however, to let Bison choose the numeric codes for all token types. Bison will automatically select codes that don't conflict with each other or with ASCII characters.
In the event that the stack type is a union, you must augment the
%token or other token declaration to include the data type alternative delimited by angle-brackets (see section More Than One Value Type).For example:
%union { /* define stack type */
double val;
symrec *tptr;
}
%token <val> NUM /* define token NUM and its type */
Operator Precedence
Use the%left, %right or %nonassoc declaration to declare a token and specify its precedence and associativity, all at once. These are called precedence declarations. See section Operator Precedence, for general information on operator precedence.The syntax of a precedence declaration is the same as that of
%token: either%left symbols...or
%left <type> symbols...And indeed any of these declarations serves the purposes of
%token. But in addition, they specify the associativity and relative precedence for all the symbols:- The associativity of an operator op determines how repeated uses of the operator nest: whether `x op y op z' is parsed by grouping x with y first or by grouping y with z first.
%leftspecifies left-associativity (grouping x with y first) and%rightspecifies right-associativity (grouping y with z first).%nonassocspecifies no associativity, which means that `x op y op z' is considered a syntax error. - The precedence of an operator determines how it nests with other operators. All the tokens declared in a single precedence declaration have equal precedence and nest together according to their associativity. When two tokens declared in different precedence declarations associate, the one declared later has the higher precedence and is grouped first.
The Collection of Value Types
The%union declaration specifies the entire collection of possible data types for semantic values. The keyword %union is followed by a pair of braces containing the same thing that goes inside a union in C.For example:
%union {
double val;
symrec *tptr;
}
This says that the two alternative types are double and symrec *. They are given names val and tptr; these names are used in the %token and %type declarations to pick one of the types for a terminal or nonterminal symbol (see section Nonterminal Symbols).Note that, unlike making a
union declaration in C, you do not write a semicolon after the closing brace.Nonterminal Symbols
When you use%union to specify multiple value types, you must declare the value type of each nonterminal symbol for which values are used. This is done with a %type declaration, like this:%type <type> nonterminal...Here nonterminal is the name of a nonterminal symbol, and type is the name given in the
%union to the alternative that you want (see section The Collection of Value Types). You can give any number of nonterminal symbols in the same %type declaration, if they have the same value type. Use spaces to separate the symbol names.Suppressing Conflict Warnings
Bison normally warns if there are any conflicts in the grammar (see section Shift/Reduce Conflicts), but most real grammars have harmless shift/reduce conflicts which are resolved in a predictable way and would be difficult to eliminate. It is desirable to suppress the warning about these conflicts unless the number of conflicts changes. You can do this with the%expect declaration.The declaration looks like this:
%expect nHere n is a decimal integer. The declaration says there should be no warning if there are n shift/reduce conflicts and no reduce/reduce conflicts. The usual warning is given if there are either more or fewer conflicts, or if there are any reduce/reduce conflicts.
In general, using
%expect involves these steps:- Compile your grammar without
%expect. Use the `-v' option to get a verbose list of where the conflicts occur. Bison will also print the number of conflicts. - Check each of the conflicts to make sure that Bison's default resolution is what you really want. If not, rewrite the grammar and go back to the beginning.
- Add an
%expectdeclaration, copying the number n from the number which Bison printed.
The Start-Symbol
Bison assumes by default that the start symbol for the grammar is the first nonterminal specified in the grammar specification section. The programmer may override this restriction with the%start declaration as follows:%start symbol
A Pure (Reentrant) Parser
A reentrant program is one which does not alter in the course of execution; in other words, it consists entirely of pure (read-only) code. Reentrancy is important whenever asynchronous execution is possible; for example, a nonreentrant program may not be safe to call from a signal handler. In systems with multiple threads of control, a nonreentrant program must be called only within interlocks.The Bison parser is not normally a reentrant program, because it uses statically allocated variables for communication with
yylex. These variables include yylval and yylloc.The Bison declaration
%pure_parser says that you want the parser to be reentrant. It looks like this:%pure_parserThe effect is that the two communication variables become local variables in
yyparse, and a different calling convention is used for the lexical analyzer function yylex. See section Calling for Pure Parsers, for the details of this. The variable yynerrs also becomes local in yyparse (see section The Error Reporting Function yyerror). The convention for calling yyparse itself is unchanged.Bison Declaration Summary
Here is a summary of all Bison declarations:%union- Declare the collection of data types that semantic values may have (see section The Collection of Value Types).
%token- Declare a terminal symbol (token type name) with no precedence or associativity specified (see section Token Type Names).
%right- Declare a terminal symbol (token type name) that is right-associative (see section Operator Precedence).
%left- Declare a terminal symbol (token type name) that is left-associative (see section Operator Precedence).
%nonassoc- Declare a terminal symbol (token type name) that is nonassociative (using it in a way that would be associative is a syntax error) (see section Operator Precedence).
%type- Declare the type of semantic values for a nonterminal symbol (see section Nonterminal Symbols).
%start- Specify the grammar's start symbol (see section The Start-Symbol).
%expect- Declare the expected number of shift-reduce conflicts (see section Suppressing Conflict Warnings).
%pure_parser- Request a pure (reentrant) parser program (see section A Pure (Reentrant) Parser).
Multiple Parsers in the Same Program
Most programs that use Bison parse only one language and therefore contain only one Bison parser. But what if you want to parse more than one language with the same program? Then you need to avoid a name conflict between different definitions ofyyparse, yylval, and so on.The easy way to do this is to use the option `-p prefix' (see section Invoking Bison). This renames the interface functions and variables of the Bison parser to start with prefix instead of `yy'. You can use this to give each parser distinct names that do not conflict.
The precise list of symbols renamed is
yyparse, yylex, yyerror, yylval, yychar and yydebug. For example, if you use `-p c', the names become cparse, clex, and so on.All the other variables and macros associated with Bison are not renamed. These others are not global; there is no conflict if the same name is used in different parsers. For example,
YYSTYPE is not renamed, but defining this in different ways in different parsers causes no trouble (see section Data Types of Semantic Values).The `-p' option works by adding macro definitions to the beginning of the parser source file, defining
yyparse as prefixparse, and so on. This effectively substitutes one name for the other in the entire parser file.Source: http://www.slac.stanford.edu/comp/unix/gnu-info/bison_1.html
