Fine tune table-driven tokenizing.

This commit is contained in:
Stephen Chung
2023-03-15 17:22:11 +08:00
parent 2aa7b99d1e
commit 41636eac55
11 changed files with 351 additions and 159 deletions

7
src/tools/README.md Normal file
View File

@@ -0,0 +1,7 @@
Build Tools
===========
| File | Description |
| -------------- | ------------------------------------------- |
| `keywords.txt` | Input file for GNU gperf for the tokenizer. |
| `reserved.txt` | Input file for GNU gperf for the tokenizer. |