8ef9aed096Switching from various Byte and Rune handlers to single Char handlers. The Char handlers determine on their own if they should handle things in byte or rune mode.Maurice Makaay2019-07-29 09:45:25 +0000
e0b1039abdMade a big jump in performance on big files with lots of comments, by reading in chunks till end of line, instead of byte-by-byte.Maurice Makaay2019-07-28 23:50:58 +0000
53ae659ef6Moving results to their own light weight tokenize.API.Result.Maurice Makaay2019-07-28 22:35:33 +0000
eda71f304eDropped PeekWithResult(), because it does not add any substantial performance. A simpler API which is virtually as fast wins any day.Maurice Makaay2019-07-27 12:26:02 +0000
fcdd3d4ea7Wow, going nicely! Some more miliseconds stripped.Maurice Makaay2019-07-26 22:56:12 +0000
62cd84bb74Use zero-indexed cursor positioning data inside stackframes. This simplifies some things. Also a bit of code cleanup.Maurice Makaay2019-07-24 10:34:24 +0000
802701ade5Added multi-byte peeks for some performance improvements.Maurice Makaay2019-07-23 23:23:40 +0000
a968f22d45Code cleanup, making the byte and rune inputs look as much the same as possible and get rid of some unneeded functionality.Maurice Makaay2019-07-23 08:03:16 +0000
dd1159e309Committing a bit of code cleanup before trying something bigger.Maurice Makaay2019-07-22 07:57:05 +0000
183f5df00dBrought back some lost performance. Doing everything via api.Input/Output causes an extra level of indirection and it does not cost that much, but we do loose performance through that route. So added private methods for the API struct, which are used internally to squeeze out a bit of extra performance.Maurice Makaay2019-07-20 23:51:08 +0000
acdf83332bUse pointers instead of values, since we're updating the structs.Maurice Makaay2019-07-20 11:50:36 +0000
0c057e4a9aSplit up the api.go into three files: api.go, api_input.go and api_output.go. This makes it easier to manage the individual code sets.Maurice Makaay2019-07-20 00:48:11 +0000
93c75af87fMoved Input and Output related fields from the API to their respective sub-structs.Maurice Makaay2019-07-20 00:28:37 +0000
7d2d8dbed3Moved input-related functions to their own API.Input struct.Maurice Makaay2019-07-19 23:41:15 +0000
9d98c9dff7Moving output functions to its own substruct of the API.Maurice Makaay2019-07-19 22:57:06 +0000
458d6f60a6A nice performance gain by making a difference between AcceptRunes/AcceptBytes and the new simpler AcceptRune/AcceptByte functions. The simpler versions are faster when only accepting a single byte or rune (which is the case in most situations).Maurice Makaay2019-07-19 21:13:15 +0000
31055a3cd3Bugfix for parsekit.read: when filling the buffer, the read offset was not taken into account for determining how many bytes could be read.Maurice Makaay2019-07-19 10:13:32 +0000
3f9c745ac4Unit tests improved for the parsekit.read package.Maurice Makaay2019-07-19 09:50:42 +0000
22bcf4677eSome work on simlifying the reader code, to see if I can squeeze some more performance out of that part.Maurice Makaay2019-07-19 08:47:13 +0000
1771e237c0Switched to a []byte backing store instead of []rune for collecting input data (we can use both bytes and runes for input in an easy way now)Maurice Makaay2019-07-18 09:26:11 +0000
b9eeac3480Work in progress on switching to byte stack. Committing to do some performance checks against master.Maurice Makaay2019-07-18 08:06:26 +0000
e659380a5fImplemented an efficient M.DropUntilEndOfLine handler, which is now used in the TOML parser for a dramatic speed increase on comment parsing.Maurice Makaay2019-07-17 23:51:37 +0000
64f92696b2Fixed unit tests for the new allocation behavior.Maurice Makaay2019-07-17 23:03:14 +0000
0a4e44b8f8Allow for bufio Readers that deliver data in chunks (like our unit test Reader)Maurice Makaay2019-07-17 23:03:00 +0000
6d3eacdcaeAllocate read buffer in 1024 byte chunks, and read the data in chunks as well. This is more efficient than reading byte by byte.Maurice Makaay2019-07-17 22:12:37 +0000
5e3e4b0f0aYay! First version for which parsing long.toml drops below 100ms! Got an outcome of 93ms. Almost down to BurntSushi's speed level, but still with a generic parser backing. Looking good!!Maurice Makaay2019-07-16 23:34:01 +0000
ddd0ed49f6Don't resize the stack slices, since we keep track of their starts and ends anyway.Maurice Makaay2019-07-16 12:19:50 +0000
06faabdfe2Small bugfix for the rune-to-byte-fallback code and added byte-support to the Str and StrNoCase matchers.Maurice Makaay2019-07-16 07:35:06 +0000
4cfdbafa6eFurther switching to byte-based input handling.Maurice Makaay2019-07-16 07:05:10 +0000
0362763e83Switched to byte input for built-in tokenize.Handler functions.Maurice Makaay2019-07-15 22:48:00 +0000
d4492e4f0aBytes reader working, now carry on switching to byte reading in the tokenizer code.Maurice Makaay2019-07-15 20:03:05 +0000
17935b7534Further performance optimization and code cleanup.Maurice Makaay2019-07-12 21:32:40 +0000
56b8df3aabRemoved loop protection code. This is useful, but it puts a performance burden on the code when doing it by keeping track of actual callers through the call stack. Maybe to be reintroduced in a future version with something like a simple counter and a maximum depth-style protection.Maurice Makaay2019-07-12 12:33:18 +0000
09746c0d2eSpeeding up the code some more. Big step was made by simplifying the cursor, continuing with that in the next commit.Maurice Makaay2019-07-12 08:02:04 +0000
92e6eec7f3implemented Cursor.moveByRune(), to get rid of some useless rune->string conversion for updating cursor positions.Maurice Makaay2019-06-30 10:16:46 +0000
4b0309453fAdded a feature to run the parser without any of the built-in sanity checks (like loop checks). This improved performance, but at the risk of missing some runtime issues with the parser implementation.Maurice Makaay2019-06-30 01:05:54 +0000
2293627232Small code cleanup things, mainly backing up the changes.Maurice Makaay2019-06-18 15:46:09 +0000
99654c2f9eSimplified some internal code, which also fixes a bug with correct error reporting from within parsekit in various edge cases.Maurice Makaay2019-06-17 13:59:31 +0000
1a280233b0Got rid of the testify dependency. My testing needs are so basic, that there's no need for this full fledged testing library.Maurice Makaay2019-06-12 15:25:15 +0000
27c97ae902Big overhaul on separating packages for code containment.Maurice Makaay2019-06-12 14:30:46 +0000
1f0e0fcc17Splitting up functionality in packages, intermediate step.Maurice Makaay2019-06-11 22:23:30 +0000
0f7b4e0d26Added a few syntactic sugar methods for ParseHandler.Maurice Makaay2019-06-11 09:09:41 +0000
65895ac502Making parsekit.reader both simpler and more complex (more complex by adopting some buffer allocation logic from the built-in bytes package, to not be copying memory all the time during the read operations.Maurice Makaay2019-06-09 21:55:01 +0000
9656cd4449The parsekit.reader.Reader now caches error messages that are returned from the embedded io.Reader. When an error is returned, the read offset and the error are stored. When later on, the same of a higher offset is requested, the error is returned again. This way the code will work for Readers that do not repeatedly return the correct error when calling the Read() method multiple times arter a first error has occurred.Maurice Makaay2019-06-09 19:42:20 +0000
76336e883eRemoved the use of Error.Full(). The default Error() method now includes the extra data from Full() (line and column offset)Maurice Makaay2019-06-09 15:20:44 +0000
add28feb33In the spirit of Go, slimmed down the ParseAPI interface. I'm no longer using ParseAPI.On(..).<DoSomething>(), but now it's simply ParseAPI.<DoSomething>(). I also dropped the difference between a Stay() and an Accept(). All that is possible now is ParseAPI.Peek() and ParseAPI.Accept().Maurice Makaay2019-06-09 10:25:49 +0000
05ae55c487Brought the examples up-to-date with the lateset code. All are working correctly now.Maurice Makaay2019-06-07 16:20:32 +0000
40bad51064Improvement a few TokanHandlers by letting them make use of the new MatchRuneByCallback method, instead of having them implement their own logic.Maurice Makaay2019-06-07 15:57:53 +0000
9a5bf8b9afFurther code cleaning for the interaction between ParseAPI and TokenAPI. Extra atoms added, also one based on a callback which can accept single runes based on thhat callback function.Maurice Makaay2019-06-07 15:48:49 +0000
6d92e1dc68Merged functionality of p.Expects(string) and p.UnexpectedInput(). It is now simply p.UnexpectedInput(string). This makes the naming of unexpected input not as magical, but explicit (which is a GoodThing). With one of the earlier incarnations of parsekit it did make sense, but it went in a way in which explicit is more idiomatic for the package.Maurice Makaay2019-06-07 07:56:24 +0000
3094b09284Adding documentation and getting the interactions between ParseAPI and TokenAPI cleaned up a bit.Maurice Makaay2019-06-07 07:26:41 +0000
3d791233e0Added a lot of IP-address-related TokenHandlers, so we can now process IPv4 addresses, IPv6 addresses, CIDR netmasks, IPv4 dotted quad netmasks, IPv4Net (ipv4 + mask) and IPv6Mask (ipv6 + mask).Maurice Makaay2019-06-05 22:16:09 +0000
05585db341Normalizing error handling, to always include the caller location in errors. This makes debugging a lot easier for users of the package, because it doesn't say stuff like 'Method() was called incorrectly', but instead something like 'Method() was called incorrectlty at /path/to/file.go:1234'.Maurice Makaay2019-06-05 10:07:50 +0000
75373e5ed5Big simplification run once more, cleaned up code, added tests and examples, made stuff unexported where possible, to slim down the exported interface.Maurice Makaay2019-06-04 23:15:02 +0000
4580962fb8Backup a load of work on typed token support, making it easy to produce tokens directly from parser/combinator-based parsing rules.Maurice Makaay2019-06-04 00:03:08 +0000
21f1aa597cMade the panic() calls (which basically indicate parser implementation bugs) more useful by referencing from where illegal calls were made.Maurice Makaay2019-05-29 07:24:27 +0000
11883b06acAdded a unit test for the actual parser loop issue that I ran into myself. This one will not bite me again!Maurice Makaay2019-05-28 23:13:28 +0000
d31d09abf0Added crude loop protection to the parser, which should prevent parsers running in circles (happened to me a few times too).Maurice Makaay2019-05-28 23:01:23 +0000
7aff3fc43eAdded a nice example that shows how a []string-based type can be turned into a parser that fills its own slice elements during parsing.Maurice Makaay2019-05-28 14:38:04 +0000
2d851103e5Cleanup of stuff that I don't need anymore, because it has been fully deprecated. Also added some tests for panic() calls in parsekit, which brings test coverage to 100%. It's not a goal as such, but it's good to know that I got there without cheaty tests :)Maurice Makaay2019-05-28 13:41:58 +0000
3dfa99c965Modified all examples and tests to make use of the new ideas on how to keep parsing state. After this commit, I can cleanup a lot of stuff from the emitting loop-based parser which was basically crap for complex parsers.Maurice Makaay2019-05-28 10:42:46 +0000
980c18099eA small change to the computation interpreter to get rid of one useless level of recursion.Maurice Makaay2019-05-28 07:26:50 +0000
Maurice Makaay