Flex is not matching my patterns in the same order that I defined them.

flex picks the rule that matches the most text (i.e., the longest possible input string). This is because flex uses an entirely different matching technique (“deterministic finite automata”) that actually does all of the matching simultaneously, in parallel. (Seems impossible, but it’s actually a fairly simple technique once you understand the principles.)

A side-effect of this parallel matching is that when the input matches more than one rule, flex scanners pick the rule that matched the most text. This is explained further in the manual, in the section See Matching.

If you want flex to choose a shorter match, then you can work around this behavior by expanding your short rule to match more text, then put back the extra:

data_.*        yyless( 5 ); BEGIN BLOCKIDSTATE;

Another fix would be to make the second rule active only during the <BLOCKIDSTATE> start condition, and make that start condition exclusive by declaring it with %x instead of %s.

A final fix is to change the input language so that the ambiguity for ‘data_’ is removed, by adding characters to it that don’t match the identifier rule, or by removing characters (such as ‘_’) from the identifier rule so it no longer matches ‘data_’. (Of course, you might also not have the option of changing the input language.)