11.3 Strings are Destroyed

My parser seems to destroy old strings, or maybe it loses track of them. Instead of reporting ‘"foo", "bar"’, it reports ‘"bar", "bar"’, or even ‘"foo\nbar", "bar"’.

This error is probably the single most frequent “bug report” sent to Bison lists, but is only concerned with a misunderstanding of the role of the scanner. Consider the following Lex code:

%{
#include <stdio.h>
char *yylval = NULL;
%}

%%
.*    yylval = yytext; return 1;
\n    /* IGNORE */
%%

int
main ()
{
  /* Similar to using $1, $2 in a Bison action.  */
  char *fst = (yylex (), yylval);
  char *snd = (yylex (), yylval);
  printf ("\"%s\", \"%s\"\n", fst, snd);
  return 0;
}

If you compile and run this code, you get:

$ flex -osplit-lines.c split-lines.l
$ gcc  -osplit-lines   split-lines.c -ll
$ printf 'one\ntwo\n' | ./split-lines
"one
two", "two"

this is because yytext is a buffer provided for reading in the action, but if you want to keep it, you have to duplicate it (e.g., using strdup). Note that the output may depend on how your implementation of Lex handles yytext. For instance, when given the Lex compatibility option -l (which triggers the option ‘%array’) Flex generates a different behavior:

$ flex -l -osplit-lines.c split-lines.l
$ gcc     -osplit-lines   split-lines.c -ll
$ printf 'one\ntwo\n' | ./split-lines
"two", "two"