 Maurice Makaay
|
53ae659ef6
|
Moving results to their own light weight tokenize.API.Result.
|
2019-07-28 22:35:33 +00:00 |
|
Maurice Makaay
|
eda71f304e
|
Dropped PeekWithResult(), because it does not add any substantial performance. A simpler API which is virtually as fast wins any day.
|
2019-07-27 12:26:02 +00:00 |
|
Maurice Makaay
|
fcdd3d4ea7
|
Wow, going nicely! Some more miliseconds stripped.
|
2019-07-26 22:56:12 +00:00 |
|
Maurice Makaay
|
daf3b9838f
|
Backup work on dropping forking support.
|
2019-07-26 14:51:40 +00:00 |
|
Maurice Makaay
|
4c94374107
|
Getting rid of forking, the new system delivers more performance.
|
2019-07-26 12:14:15 +00:00 |
|
Maurice Makaay
|
87cdadae78
|
Hmm... this whole snapshot idea seems to work and a valid replacement for the forking method.
|
2019-07-26 08:02:37 +00:00 |
|
Maurice Makaay
|
bc9e718e47
|
Lowering the number of forks required.
|
2019-07-24 22:42:40 +00:00 |
|
Maurice Makaay
|
99b0abc490
|
WIP on lowering the number of forks required.
|
2019-07-24 22:42:16 +00:00 |
|
Maurice Makaay
|
548289560b
|
Code cleanup.
|
2019-07-24 11:03:02 +00:00 |
|
Maurice Makaay
|
62cd84bb74
|
Use zero-indexed cursor positioning data inside stackframes. This simplifies some things.
Also a bit of code cleanup.
|
2019-07-24 10:34:24 +00:00 |
|
Maurice Makaay
|
802701ade5
|
Added multi-byte peeks for some performance improvements.
|
2019-07-23 23:23:40 +00:00 |
|
Maurice Makaay
|
7037c6d24a
|
Fixing some naming inconsistencies.
|
2019-07-23 17:55:13 +00:00 |
|
Maurice Makaay
|
a968f22d45
|
Code cleanup, making the byte and rune inputs look as much the same as possible and get rid of some unneeded functionality.
|
2019-07-23 08:03:16 +00:00 |
|
Maurice Makaay
|
93d2cfa6f1
|
Backup work.
|
2019-07-22 23:28:05 +00:00 |
|
Maurice Makaay
|
cf679b2225
|
Backup work for next refactoring step.
|
2019-07-22 22:16:28 +00:00 |
|
Maurice Makaay
|
070e6a13a7
|
Made some nice steps, backup and continue!
|
2019-07-22 15:37:52 +00:00 |
|
Maurice Makaay
|
dd1159e309
|
Committing a bit of code cleanup before trying something bigger.
|
2019-07-22 07:57:05 +00:00 |
|
Maurice Makaay
|
183f5df00d
|
Brought back some lost performance. Doing everything via api.Input/Output causes an extra level of indirection and it does not cost that much, but we do loose performance through that route. So added private methods for the API struct, which are used internally to squeeze out a bit of extra performance.
|
2019-07-20 23:51:08 +00:00 |
|
Maurice Makaay
|
acdf83332b
|
Use pointers instead of values, since we're updating the structs.
|
2019-07-20 11:50:36 +00:00 |
|
Maurice Makaay
|
7998d05113
|
More efficient version of MatchOctet.
|
2019-07-20 01:50:12 +00:00 |
|
Maurice Makaay
|
0c057e4a9a
|
Split up the api.go into three files: api.go, api_input.go and api_output.go. This makes it easier to manage the individual code sets.
|
2019-07-20 00:48:11 +00:00 |
|
Maurice Makaay
|
93c75af87f
|
Moved Input and Output related fields from the API to their respective sub-structs.
|
2019-07-20 00:28:37 +00:00 |
|
Maurice Makaay
|
7d2d8dbed3
|
Moved input-related functions to their own API.Input struct.
|
2019-07-19 23:41:15 +00:00 |
|
Maurice Makaay
|
9d98c9dff7
|
Moving output functions to its own substruct of the API.
|
2019-07-19 22:57:06 +00:00 |
|
Maurice Makaay
|
458d6f60a6
|
A nice performance gain by making a difference between AcceptRunes/AcceptBytes and the new simpler AcceptRune/AcceptByte functions. The simpler versions are faster when only accepting a single byte or rune (which is the case in most situations).
|
2019-07-19 21:13:15 +00:00 |
|
Maurice Makaay
|
9a53ea9012
|
Working on API speed.
|
2019-07-19 14:44:44 +00:00 |
|
Maurice Makaay
|
31055a3cd3
|
Bugfix for parsekit.read: when filling the buffer, the read offset was not taken into account for determining how many bytes could be read.
|
2019-07-19 10:13:32 +00:00 |
|
Maurice Makaay
|
3f9c745ac4
|
Unit tests improved for the parsekit.read package.
|
2019-07-19 09:50:42 +00:00 |
|
Maurice Makaay
|
22bcf4677e
|
Some work on simlifying the reader code, to see if I can squeeze some more performance out of that part.
|
2019-07-19 08:47:13 +00:00 |
|
Maurice Makaay
|
1771e237c0
|
Switched to a []byte backing store instead of []rune for collecting input data (we can use both bytes and runes for input in an easy way now)
|
2019-07-18 09:26:11 +00:00 |
|
Maurice Makaay
|
b9eeac3480
|
Work in progress on switching to byte stack. Committing to do some performance checks against master.
|
2019-07-18 08:06:26 +00:00 |
|
Maurice Makaay
|
e659380a5f
|
Implemented an efficient M.DropUntilEndOfLine handler, which is now used in the TOML parser for a dramatic speed increase on comment parsing.
|
2019-07-17 23:51:37 +00:00 |
|
Maurice Makaay
|
64f92696b2
|
Fixed unit tests for the new allocation behavior.
|
2019-07-17 23:03:14 +00:00 |
|
Maurice Makaay
|
0a4e44b8f8
|
Allow for bufio Readers that deliver data in chunks (like our unit test Reader)
|
2019-07-17 23:03:00 +00:00 |
|
Maurice Makaay
|
6d3eacdcae
|
Allocate read buffer in 1024 byte chunks, and read the data in chunks as well. This is more efficient than reading byte by byte.
|
2019-07-17 22:12:37 +00:00 |
|
Maurice Makaay
|
5e3e4b0f0a
|
Yay! First version for which parsing long.toml drops below 100ms! Got an outcome of 93ms. Almost down to BurntSushi's speed level, but still with a generic parser backing. Looking good!!
|
2019-07-16 23:34:01 +00:00 |
|
Maurice Makaay
|
ddd0ed49f6
|
Don't resize the stack slices, since we keep track of their starts and ends anyway.
|
2019-07-16 12:19:50 +00:00 |
|
Maurice Makaay
|
06faabdfe2
|
Small bugfix for the rune-to-byte-fallback code and added byte-support to the Str and StrNoCase matchers.
|
2019-07-16 07:35:06 +00:00 |
|
Maurice Makaay
|
4cfdbafa6e
|
Further switching to byte-based input handling.
|
2019-07-16 07:05:10 +00:00 |
|
Maurice Makaay
|
0362763e83
|
Switched to byte input for built-in tokenize.Handler functions.
|
2019-07-15 22:48:00 +00:00 |
|
Maurice Makaay
|
d4492e4f0a
|
Bytes reader working, now carry on switching to byte reading in the tokenizer code.
|
2019-07-15 20:03:05 +00:00 |
|
Maurice Makaay
|
17935b7534
|
Further performance optimization and code cleanup.
|
2019-07-12 21:32:40 +00:00 |
|
Maurice Makaay
|
56b8df3aab
|
Removed loop protection code. This is useful, but it puts a performance burden on the code when doing it by keeping track of actual callers through the call stack. Maybe to be reintroduced in a future version with something like a simple counter and a maximum depth-style protection.
|
2019-07-12 12:33:18 +00:00 |
|
Maurice Makaay
|
09746c0d2e
|
Speeding up the code some more. Big step was made by simplifying the cursor, continuing with that in the next commit.
|
2019-07-12 08:02:04 +00:00 |
|
Maurice Makaay
|
7116aa47df
|
Squishing out more performance.
|
2019-07-12 00:21:02 +00:00 |
|
Maurice Makaay
|
a4eda45d2c
|
Made all unit tests work again.
|
2019-07-11 14:55:08 +00:00 |
|
Maurice Makaay
|
3c9a678d7a
|
Fixed the ModifyDrop() behavior. It worked, but it caused memory build-up in the old implementation.
|
2019-07-11 14:52:12 +00:00 |
|
Maurice Makaay
|
c532af67ca
|
Optimization round completed (for now :-) All tests successful.
|
2019-07-11 12:43:57 +00:00 |
|
Maurice Makaay
|
7598b62dd0
|
Finalized the work-through of the new version of the tokenizer code.
|
2019-07-10 20:36:21 +00:00 |
|
Maurice Makaay
|
48d7fda9f8
|
New implementation for performance.
|
2019-07-10 11:26:47 +00:00 |