Java.io.StreamTokenizer Class in Java | Set 1 - GeeksforGeeks

09 Jan, 2017
Java.io.StreamTokenizer class parses input stream into “tokens”.It allows to read one token at a time. Stream Tokenizer can recognize numbers, quoted strings, and various comment styles.Declaration :
public class StreamTokenizer
  extends Object
Constructor :StreamTokenizer(Reader arg) : Creates a tokenizer that parses the given character stream.
Methods :
commentChar : java.io.StreamTokenizer.commentChar(int arg) ignores all characters from the single-line comment character to the end of the line by this StreamTokenizer.Syntax :public void commentChar(int arg) 
Parameters : 
arg : the character after which all characters are ignored in the line.
Return :
No value is returned.Implementation :// Java Program illustrating use of commentChar() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,                                               FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);        // Use of commentChar() method        token.commentChar('a');        int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :Programmers123GeeksHelloa Program is explained here my friends.Output:Word : Progr
Number : 1.0
Number : 2.0
Number : 3.0
Word : Geeks
Word : Hello

public void commentChar(int arg) 
Parameters : 
arg : the character after which all characters are ignored in the line.
Return :
No value is returned.
Implementation :
// Java Program illustrating use of commentChar() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,                                               FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);        // Use of commentChar() method        token.commentChar('a');        int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
Programmers123GeeksHelloa Program is explained here my friends.
Output:
Word : Progr
Number : 1.0
Number : 2.0
Number : 3.0
Word : Geeks
Word : Hello

lineno() : java.io.StreamTokenizer.lineno() returns the current line number of this StreamTokenizer.Syntax :public int lineno()
Parameters : 
arg : the character after which all characters are ignored in the line.
Return :
returns the current line number of this StreamTokenizer.Implementation :// Java Program  illustrating use of lineno() method  import java.io.*;public class NewClass{      public static void main(String[] args) throws InterruptedException,                                                   FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);                  token.eolIsSignificant(true);        // Use of lineno() method         // to get current line no.        System.out.println("Line Number:" + token.lineno());          token.commentChar('a');        int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_EOL:                System.out.println("");                System.out.println("Line No. : " + token.lineno());                break;            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Output:Line Number:1
Word : Progr

Line No. : 2
Number : 1.0

Line No. : 3
Number : 2.0

Line No. : 4
Number : 3.0

Line No. : 5
Word : Geeks

Line No. : 6
Word : Hello

Line No. : 7
Word : This
Word : is

public int lineno()
Parameters : 
arg : the character after which all characters are ignored in the line.
Return :
returns the current line number of this StreamTokenizer.
Implementation :
// Java Program  illustrating use of lineno() method  import java.io.*;public class NewClass{      public static void main(String[] args) throws InterruptedException,                                                   FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);                  token.eolIsSignificant(true);        // Use of lineno() method         // to get current line no.        System.out.println("Line Number:" + token.lineno());          token.commentChar('a');        int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_EOL:                System.out.println("");                System.out.println("Line No. : " + token.lineno());                break;            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Output:
Line Number:1
Word : Progr

Line No. : 2
Number : 1.0

Line No. : 3
Number : 2.0

Line No. : 4
Number : 3.0

Line No. : 5
Word : Geeks

Line No. : 6
Word : Hello

Line No. : 7
Word : This
Word : is

toString() : java.io.StreamTokenizer.toString() represents current Stream token as a string along with it’s line no.Syntax :public String toString()
Return :
represents current Stream token as a string along with it's line no.Implementation :// Java Program  illustrating use of toString() method   import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,                                               FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);                  int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                // Value of ttype field returned by nextToken()                 System.out.println("Number : " + token.nval);                break;                // Value of ttype field returned by nextToken()            case StreamTokenizer.TT_WORD:                                  // Use of toStringn() method                System.out.println("Word : " + token.toString());                break;             }        }    }}Output:Word : Token[Programmers], line 1
Number : 1.0
Number : 2.0
Number : 3.0
Word : Token[Geeks], line 5
Word : Token[Hello], line 6
Word : Token[a], line 7
Word : Token[Program], line 7
Word : Token[is], line 7
Word : Token[explained], line 7
Word : Token[here], line 7
Word : Token[my], line 7
Word : Token[friends.], line 7

public String toString()
Return :
represents current Stream token as a string along with it's line no.
Implementation :
// Java Program  illustrating use of toString() method   import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,                                               FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);                  int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                // Value of ttype field returned by nextToken()                 System.out.println("Number : " + token.nval);                break;                // Value of ttype field returned by nextToken()            case StreamTokenizer.TT_WORD:                                  // Use of toStringn() method                System.out.println("Word : " + token.toString());                break;             }        }    }}
Output:
Word : Token[Programmers], line 1
Number : 1.0
Number : 2.0
Number : 3.0
Word : Token[Geeks], line 5
Word : Token[Hello], line 6
Word : Token[a], line 7
Word : Token[Program], line 7
Word : Token[is], line 7
Word : Token[explained], line 7
Word : Token[here], line 7
Word : Token[my], line 7
Word : Token[friends.], line 7

eolIsSignificant() : java.io.StreamTokenizer.eolIsSignificant(boolean arg) tells whether to treat end of line as a token or not. If ‘arg’ is true, then it End Of Line is treated as a token. If true, then the method returns TT_EOL and set the ttype field when End Of Line is reached.If ‘arg’ is false then the End Of Line is treated simply as a white space.Syntax :public void eolIsSignificant(boolean arg)
Parameters :
arg : boolean which tells whether to take EOL as a token or white space
Return :
No value is returned.Implementation :// Java Program  illustrating use of eolIsSignificant() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          boolean arg = true;        // Use of eolIsSignificant() method        token.eolIsSignificant(arg);        // Here the 'arg' is set true, so EOL is treated as a token          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_EOL:                System.out.println("End of Line encountered.");                break;            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :1Geeks2For3GeeksOutput :Number : 1.0
End of Line encountered.
Word : Geeks
End of Line encountered.
Number : 2.0
End of Line encountered.
Word : For
End of Line encountered.
Number : 3.0
End of Line encountered.
Word : Geeks
public void eolIsSignificant(boolean arg)
Parameters :
arg : boolean which tells whether to take EOL as a token or white space
Return :
No value is returned.
Implementation :
// Java Program  illustrating use of eolIsSignificant() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          boolean arg = true;        // Use of eolIsSignificant() method        token.eolIsSignificant(arg);        // Here the 'arg' is set true, so EOL is treated as a token          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_EOL:                System.out.println("End of Line encountered.");                break;            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
1Geeks2For3Geeks
Output :
Number : 1.0
End of Line encountered.
Word : Geeks
End of Line encountered.
Number : 2.0
End of Line encountered.
Word : For
End of Line encountered.
Number : 3.0
End of Line encountered.
Word : Geeks
nextToken() : java.io.StreamTokenizer.nextToken() parses the next token from the Input Stream and returns it’s value to the ttype field along with the additional fields like ‘sval’, ‘nval’.Syntax :public int nextToken()
Parameters :
------
Return :
value to the ttype fieldImplementation :// Java Program  illustrating use of nextToken() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of nextToken() method to parse Next Token from the Input Stream        int t = token.nextToken();        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :1This program tells2about use of3next token() methodOutput :Word : This
Word : program
Word : tells
Number : 2.0
Word : about
Word : use
Word : of
Number : 3.0
Word : next
Word : token
Word : method
public int nextToken()
Parameters :
------
Return :
value to the ttype field
Implementation :
// Java Program  illustrating use of nextToken() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of nextToken() method to parse Next Token from the Input Stream        int t = token.nextToken();        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
1This program tells2about use of3next token() method
Output :
Word : This
Word : program
Word : tells
Number : 2.0
Word : about
Word : use
Word : of
Number : 3.0
Word : next
Word : token
Word : method
lowerCaseMode() : java.io.StreamTokenizer.lowerCaseMode(boolean arg) tells whether to lowercase the word tokens automatically or not. If the flag – ‘arg’ is set true, then the value of ‘sval’ field is lowered.Syntax :public void lowerCaseMode(boolean arg)
Parameters :
arg : indicates whether to lowercase the word tokens automatically or not
Return :
voidImplementation :// Java Program  illustrating use of lowerCaseMode() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          /* Use of lowerCaseMode() method to           Here, the we have set the Lower Case Mode ON        */        boolean arg = true;        token.lowerCaseMode(arg);          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :Hello GeeksThis Is AboutLowerCaseMode()Output :Word : hello
Word : geeks
Word : this
Word : is
Word : about
Word : lowercasemode
public void lowerCaseMode(boolean arg)
Parameters :
arg : indicates whether to lowercase the word tokens automatically or not
Return :
void
Implementation :
// Java Program  illustrating use of lowerCaseMode() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          /* Use of lowerCaseMode() method to           Here, the we have set the Lower Case Mode ON        */        boolean arg = true;        token.lowerCaseMode(arg);          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
Hello GeeksThis Is AboutLowerCaseMode()
Output :
Word : hello
Word : geeks
Word : this
Word : is
Word : about
Word : lowercasemode
ordinaryChar() : java.io.StreamTokenizer.ordinaryChar(int arg) sets ‘arg’ character as an ordinary character. It will remove the arg character, if it has any significance as word, number, white space or comment Character.Syntax :public void ordinaryChar(int arg)
Parameters :
arg : the character which is to be set as an Ordinary Character
Return :
voidImplementation :// Java Program  illustrating use of ordinaryChar() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of ordinaryChar() method         // Here we have taken 's' as an ordinary character         token.ordinaryChar('s');          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :Hello GeeksThissss Issszz AboutordinaryChar()This method has remove ‘s’ from the entire StreamOutput :Word : Hello
Word : Geek
Word : Thi
Word : I
Word : zz
Word : About
Word : ordinaryChar
public void ordinaryChar(int arg)
Parameters :
arg : the character which is to be set as an Ordinary Character
Return :
void
Implementation :
// Java Program  illustrating use of ordinaryChar() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of ordinaryChar() method         // Here we have taken 's' as an ordinary character         token.ordinaryChar('s');          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
Hello GeeksThissss Issszz AboutordinaryChar()
This method has remove ‘s’ from the entire Stream
Output :
Word : Hello
Word : Geek
Word : Thi
Word : I
Word : zz
Word : About
Word : ordinaryChar
ordinaryChars() : java.io.StreamTokenizer.ordinaryChars(int low, int high) sets character in the range – ‘a to b’ to Ordinary character in the StreamTokenizerSyntax :public void ordinaryChars(int low, int high)
Parameters :
low : lower limit of range
high : higher limit of range
Return :
voidImplementation :// Java Program  illustrating use of ordinaryChars() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of ordinaryChars() method         // Here we have taken low = 'a' and high = 'c'          token.ordinaryChars('a','c');          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :Hello GeeksThisisaboutordinaryChars()Output :Word : Hello
Word : Geeks
Word : This
Word : is
Word : out
Word : ordin
Word : ryCh
Word : rsNext Article – Java.io.StreamTokenizer Class in Java | Set 2 This article is contributed by Mohit Gupta . If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.My Personal Notes
arrow_drop_upSave
public void ordinaryChars(int low, int high)
Parameters :
low : lower limit of range
high : higher limit of range
Return :
void
Implementation :
// Java Program  illustrating use of ordinaryChars() method  import java.io.*;public class NewClass{    public static void main(String[] args) throws InterruptedException,    FileNotFoundException, IOException    {        FileReader reader = new FileReader("ABC.txt");        BufferedReader bufferread = new BufferedReader(reader);        StreamTokenizer token = new StreamTokenizer(bufferread);          // Use of ordinaryChars() method         // Here we have taken low = 'a' and high = 'c'          token.ordinaryChars('a','c');          int t;        while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)        {            switch (t)            {            case StreamTokenizer.TT_NUMBER:                System.out.println("Number : " + token.nval);                break;            case StreamTokenizer.TT_WORD:                System.out.println("Word : " + token.sval);                break;              }        }    }}
Note :This program won’t run here as no ‘ABC’ file exists. You can check this code on Java compiler on your system.To check this code, create a file ‘ABC’ on your system.‘ABC’ file contains :
Hello GeeksThisisaboutordinaryChars()
Output :
Word : Hello
Word : Geeks
Word : This
Word : is
Word : out
Word : ordin
Word : ryCh
Word : rs
Next Article – Java.io.StreamTokenizer Class in Java | Set 2 This article is contributed by Mohit Gupta . If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Java-I/O
Java
Java
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Object Oriented Programming (OOPs) Concept in Java
HashMap in Java with Examples
Interfaces in Java
Stream In Java
How to iterate any Map in Java
ArrayList in Java
Initialize an ArrayList in Java
Stack Class in Java
Multidimensional Arrays in Java
Singleton Class in Java