MAKING A TOKENIZER IN C PROGRAMMING I am currently working on an assignment wher

Question

MAKING A TOKENIZER IN C PROGRAMMING

I am currently working on an assignment where I have to create a Tokenizer in C, the program will read in a string included in the command line and "break it up" into several categories such as "word", "decimal integer constant", etc. Here is an example of it's execution:

I must accomplish this using structures and pointers. This is my starting file: http://pastebin.com/vrHgLN42

MY MAIN QUESTION:

How do I go about chopping up the input string into mulitiple "pieces"?

Explanation / Answer

You should understand the concept of delimiters. Delimiter is a way in which the words of the entire string are split up, and concatenated to form a single sentence. So,
1. Keep capturing the alphabets in a char array called current_word

2.while parsing the moment you encounter a space/blank you should capture the current word into your main list of WORDS.
empty the array current_word

3. Continue with 1 again.

Related Questions

Navigate

• Browse (All)
• Browse M
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.