My period of disruptions is hopefully coming to an end, with this blog and our other websites now hosted on a different hardware (though the same VM), and the electrical work all finished and tidied up. So, I’ve been managing to get back into development. There’s always a bit of a slow start after a break – it’s harder to get into a state of flow, until you gradually regain the little bits of temporary detailed knowledge about the aspect of the program you’re working on.
As I mentioned at the end of my last development related blog, ‘Testing DSLs and Code Generators‘, I’ve been working on getting my two interpreters into a more complete and stable state. Or at least, I started on that, but then got slightly sidetracked. In my parser rule definitions I had added a way of triggering a specific error or warning is an expected token was missing, so a part of a rule like ';'!5 would mean, a semicolon is expected, but if it’s not there issue warning 5 and try and continue. If there were two exclamation marks instead of one, the meaning would be issue an error, and stop attempting to parse the input. In my template language there are two distinct types of variable, the data that comes from the DSL parser, which is treated as constants, and the template language variables. Because the template doesn’t define what the input from the parser is, I wanted a way of ensuring these two sets of variables could never clash, so I’ve decided the template language ones will use PHP/Perl style naming, with a leading $. This is a newish decision, so I had to go back and fix my parser rules and examples to match these variable names correctly. At this point I also tried use the !! rule option, but I misremembered it as “predicted incorrect token”, rather than “predicted missing token” – after a few minutes confusion I realised what I’d done, but that got me thinking. I chose the ! symbol on the basis of it’s writing meaning, sort of ‘take note’, but in programming it often means logical NOT, and so it feels more natural for a negative rule. So, that lead to my slight sidetrack…
I’ve gone back to my parser rules, and changed the “predicted missing token” syntax to token# for a warning and token## for an error, and used the ! and !! for the more natural feeling “predicted incorrect token”. While I was doing that I also started adding look-ahead (@token) and negative look-ahead (!@token) rules. In the toy example ('a' !@'b' | 'a' 'b') 'c' run on input a b c I’m getting the correct list of tokens out, but the a is appearing twice in the output value. I’m fairly sure this is the same bug that was causing problems when I tried to implement rollback, but now showing up in a much simpler context. It’s Friday afternoon now, so I’m going to leave it for a day or two, but on Monday I’ll set about fixing that bug, and maybe getting rollback working.
Now, I’m going to play with my new stylophone-like synthesisers, and maybe make another video trying out their sounds.