Generating Tokenizers with Flat Automata

Thumbnail Image

Files

10.4204_EPTCS.370.5.pdf (169.18 KB)

Date

2022-09-20

Authors

Journal Title

Journal ISSN

Volume Title

Publisher

Open Publishing Association

Abstract

We introduce flat automata for automatic generation of tokenizers. Flat automata are a simple representation of standard finite automata. Using the flat representation, automata can be easily constructed, combined and printed. Due to the use of border functions, flat automata are more compact than standard automata in the case where intervals of characters are attached to transitions, and the standard algorithms on automata are simpler.

Description

Keywords

Flat Automata, Tokenizer Generation, Finite Automata Representation, Border Functions, Interval-Based Transitions

Citation

Hans, N., Muktubayeva, D. (2022). Generating Tokenizers with Flat Automata. Electronic Proceedings in Theoretical Computer Science. D. Della Monica and P. Ganty (Eds.): 13th International Symposium on Games, Automata, Logics and Formal Verification (GandALF 22) EPTCS 370, 2022, pp. 66–80. https://doi.org/10.4204/eptcs.370.5

URI

https://doi.org/10.4204/eptcs.370.5
 https://nur.nu.edu.kz/handle/123456789/10487

Collections

Conference Proceedings

Endorsement

Review

Supplemented By

Referenced By