ed846c7e53Wrote a first crude version of specific tokenizer handlers for string parsing. Missing feature is good error reporting from the tokenize handler code (which has been a TODO for a while, so a nice one to implement after this).Maurice Makaay2019-07-31 07:51:37 +0000
74274e04fbMade a big jump in performance on big files with lots of comments, by reading in chunks till end of line, instead of byte-by-byte.Maurice Makaay2019-07-28 23:50:30 +0000
44f022544fRestructured the BurntSushi testing code, to not let it be a part of the TOML AST.Maurice Makaay2019-07-28 14:51:14 +0000
dea3eb987bChanges for compatibliity with latest parsekit, and some speed improvements.Maurice Makaay2019-07-26 22:56:24 +0000
c67a91b1caIgnore build output, removed some profiling code.Maurice Makaay2019-06-30 10:15:39 +0000
a28269800aMake use of the new feature to run without sanity check on the parser, to make the big testfiles-based test suite faster.Maurice Makaay2019-06-30 01:07:13 +0000
9680c2b844Implemented TOML arrays as a linked list, which prevents a lot of memory copying on large arrays. Also implemented a json comparison, inspired by BurnSushi's version, which makes the tests work when keys in JSON maps are not in the expected order.Maurice Makaay2019-06-29 23:11:03 +0000
7227fdcb93Integrated multiple file-based test suites (BurntSushi, iarna and my own) into the unit tests. BurntSushi's tests are also still supported using the test interface implementation from cmd/burntsushi-tester.Maurice Makaay2019-06-29 15:45:52 +0000
ec3a47da8dComposition over inheritance, why didn't I do that in the first place I wonder.Maurice Makaay2019-06-28 08:34:35 +0000
63cbe18d03Backup work, before switching from module development setup to standard GOPATH workspace setup (some tooling simply is not yet working with it, and I mainly miss refactoring a lot)Maurice Makaay2019-06-27 07:36:24 +0000
54cb75955dWent over the full TOML AST implementation to give stuff good names. Also pushed all tests to ast_test to make sure that the public interface is sufficient for using the packageMaurice Makaay2019-06-26 15:38:32 +0000
688894dbf2Extracted the toml AST to its own subpackage. Preparation for having encoder and decoder as well, not all inside the main toml dir.Maurice Makaay2019-06-26 13:57:22 +0000
c536dd1243Huge overhaul in AST handling. The AST is now fully integrated with the parser, which has been simplified quite a bit because of this.Maurice Makaay2019-06-25 21:29:05 +0000
15560b29b0Added a good string formatter for the AST, to making testing more straight forward.Maurice Makaay2019-06-24 07:05:38 +0000
726b5a377bA bit of code cleanup, to let the date/time code better represent the ideas behind the followed methodology.Maurice Makaay2019-06-19 11:28:03 +0000
45245e05b6Some small cleanup, committing before I continue with inline lists.Maurice Makaay2019-06-19 11:09:26 +0000
c405260ea6All base value types for TOML implemented (added date/time types and implemented correct detection of the possible value types). All examples from the TOML specification key/value assignments have been added to the unit tests. All are gree :-)Maurice Makaay2019-06-19 10:42:51 +0000
c29afaeaccNope, NOW I'm done with the number types. On reading the docs, I noticed that I missed an implementation for floats 'nan' and 'inf'. All taken care of now!Maurice Makaay2019-06-18 23:13:59 +0000
e3e408dfdbPhew, that was quite the update. I've now got a working implementation of a parser/combinator-like matching API, which prevents us from having to specify everything in state functions. That is way too low level for a lot of things. I'd rather have parser/combinator-style definitions for chunks of the input and keeping the state functions for higher level document structure parsing.Maurice Makaay2019-05-19 23:35:03 +0000
55e23874f7Hmm... even beter wording! Fully from the parser writer's perspective, hiding internals.Maurice Makaay2019-05-18 13:21:56 +0000
a569c430d5More sensible state naming. No need to prefix every state function with state_.Maurice Makaay2019-05-18 01:49:30 +0000
e2e4fbd901Slightly improved routing handling to prevent repeition in code for string parsing.Maurice Makaay2019-05-18 01:28:27 +0000
4556520582Added some more straightforwardness to the pattern magic coding. I can now write stuff like p.After(upper, upper, 4hex).Store() to store runes in the string buffer when the match is complete. Other options instead of Store() are for now Backup() (making it more of a peek) or Ignore() (skipping over the scanned text). I think this methodology forms a nice mental representation for the coder that uses the library. It's close to how we think about parsing (or at least I do)Maurice Makaay2019-05-18 01:19:25 +0000
666cff3af3Ahhhh found a name that clicked for the more general layer of the parser code: parsekit. That is short and tells me what it is. It's not a parser, but something to build parsers with. Now I could also name the actual parsing code as I would like to, namely 'toml/parser'. So it feels like the structure is settling down.Maurice Makaay2019-05-17 22:03:10 +0000
3f638c59cdSome huge refactorings before I start moving forward again. Learned a lot about Go in the meanwhile, and more ideas keep popping up to improve what I've got so far even further.Maurice Makaay2019-05-17 19:56:55 +0000
9f19add210Now the parser code is out of the way, we can split up the state functions describing the syntax of TOML into separate files, while still keeping it maintainable.Maurice Makaay2019-05-17 14:13:25 +0000
db4a8f7942Removed some source files that are not used anymore, after the refactoring steps.Maurice Makaay2019-05-17 12:46:09 +0000
f86ef2b918Splitting off a more generic parser (it's fun getting to know a language, but you keep refactoring with all new stuff that you learn :-)Maurice Makaay2019-05-17 12:44:24 +0000
453a625a38Fixed cursor position tracking (to report row + column on error). All tests are green again :-)Maurice Makaay2019-05-17 00:11:12 +0000
29a13834ddSimplify, simplify, simplify, and make handling of invalid UTF8 or unexpected en of file more robust.Maurice Makaay2019-05-16 23:26:43 +0000
dc47ac3b71Make short and long UTF8 escape sequences work in strings.Maurice Makaay2019-05-16 16:17:23 +0000
cbc4f04179Code cleanup and refactoring run, both functional code and the tests.Maurice Makaay2019-05-16 14:17:06 +0000
6636a7a672Implemented a separated lexer.StringBuffer, to not pollute lexer code with string building code. The string builder can provide built strings both as literal as-is string (in TOML: single quotes), or as interpreted strings (in TOML: between double quotes)
Ubuntu
2019-05-15 22:47:06 +0000