Please complete task 3 using Scala 3, replacing your solution with the symbol ???. package words

  import org.apache.spark.rdd.RDD
  import org.apache.spark.SparkContext
  import org.apache.spark.SparkContext._


  @main def main(): Unit =

    /*
     * Let us start by setting up a Spark context which runs locally
     * using two worker threads.
     *
     * Here we go:
     *
     */

    val sc = new SparkContext("local[2]", "words")

    /*
    * The following setting controls how ``verbose'' Spark is.
    * Comment this out to see all debug messages.
    * Warning: doing so may generate massive amount of debug info,
    * and normal program output can be overwhelmed!
    */
    sc.setLogLevel("WARN") 

    /*
     * Next, let us set up our input. 
     */

    val path = "a01-words/data/"
    /*
     * After the path is configured, we need to decide which input
     * file to look at. There are two choices -- you should test your
     * code with "War and Peace" (default below), and then use the code with
     * "Wealth of Nations" to compute the correct solutions 
     * (which you will submit to A+ for grading).
     * 
     */

    // Tolstoy -- War and Peace         (test input)
    val filename = path ++ "pg2600.txt"

    // Smith   -- Wealth of Nations     (uncomment line below to use as input)
//    val filename = path ++ "pg3300.txt"

    /*
     * Now you may want to open up in a web browser 
     * the Scala programming guide for 
     * Spark version 3.3.1:
     *
     * http://spark.apache.org/docs/3.3.1/programming-guide.html
     * 
     */

    /*
     * Let us now set up an RDD from the lines of text in the file:
     *
     */

    val lines: RDD[String] = sc.textFile(filename)

    /* The following requirement sanity-checks the number of lines in the file 
     * -- if this requirement fails you are in trouble.
     */

    require((filename.contains("pg2600.txt") && lines.count() == 65007) ||
            (filename.contains("pg3300.txt") && lines.count() == 35600))

    /* 
     * Let us make one further sanity check. That is, we want to
     * count the number of lines in the file that contain the 
     * substring "rent".
     *
     */

    val lines_with_rent: RDD[String] = 
      lines.filter(line => line.contains("rent"))

    val rent_count = lines_with_rent.count()
    println("OUTPUT: \"rent\" occurs on %d lines in \"%s\""
              .format(rent_count, filename))
    require((filename.contains("pg2600.txt") && rent_count == 360) ||
            (filename.contains("pg3300.txt") && rent_count == 1443))

    /*
     * All right, if the execution continues this far without
     * failing a requirement, we should be pretty sure that we have
     * the correct file. Now we are ready for the work that you need
     * to put in. 
     *
     */

    /*
     * Spark operates by __transforming__ RDDs. For example, above we
     * took the RDD 'lines', and transformed it into the RDD 'lines_with_rent'
     * using the __filter__ transformation. 
     *
     * Important: 
     * While the code that manipulates RDDs may __look like__ we are
     * manipulating just another Scala collection, this is in fact
     * __not__ the case. An RDD is an abstraction that enables us 
     * to easily manipulate terabytes of data in a cluster computing
     * environment. In this case the dataset is __distributed__ across
     * the cluster. In fact, it is most likely that the entire dataset
     * cannot be stored in a single cluster node. 
     *
     * Let us practice our skills with simple RDD transformations. 
     *
     */

    /*
     * Task 1:
     * This task asks you to transform the RDD
     * 'lines' into an RDD 'depunctuated_lines' so that __on each line__, 
     * all occurrences of any of the punctuation characters
     * ',', '.', ':', ';', '\"', '(', ')', '{', '}' have been deleted.
     *
     * Hint: it may be a good idea to consult 
     * http://www.scala-lang.org/api/3.2.1/scala/collection/StringOps.html
     *
     */

    val depunctuated_lines: RDD[String] =
      lines.map(line => line.replaceAll("[,.:;\"(){}]", ""))


    /* 
     * Let us now check and print out data that you want to 
     * record (__when the input file is "pg3300.txt"__) into 
     * the file "wordsSolutions.scala" that you need to submit for grading
     * together with this file. 
     */
    
    val depunctuated_length = depunctuated_lines.map(_.length).reduce(_ + _)
    println("OUTPUT: total depunctuated length is %d".format(depunctuated_length))
    require(!filename.contains("pg2600.txt") || depunctuated_length == 3069444)


    /*
     * Task 2:
     * Next, let us now transform the RDD of depunctuated lines to
     * an RDD of consecutive __tokens__. That is, we want to split each
     * line into zero or more __tokens__ where a __token__ is a 
     * maximal nonempty sequence of non-space (non-' ') characters on a line. 
     * Blank lines or lines with only space (' ') in them should produce 
     * no tokens at all. 
     *
     * Hint: Use either a map or a flatMap to transform the RDD 
     * line by line. Again you may want to take a look at StringOps
     * for appropriate methods to operate on each line. Use filter
     * to get rid of blanks as necessary.
     *
     */

    val tokens: RDD[String] = depunctuated_lines.flatMap(line => line.split("\\s+")).filter(token => token.nonEmpty)
                            // transform 'depunctuated_lines' to tokens


    /* ... and here comes the check and the printout. */

    val token_count = tokens.count()
    println("OUTPUT: %d tokens".format(token_count))
    require(!filename.contains("pg2600.txt") || token_count == 566315)


    /*
     * Task 3:
     * Transform the RDD of tokens into a new RDD where all upper case 
     * characters in each token get converted into lower case. Here you may 
     * restrict the conversion to characters in the Roman alphabet 
     * 'A', 'B', ..., 'Z'.
     *
     */

    val tokens_lc: RDD[String] = ???
                            // map each token in 'tokens' to lower case


    /* ... and here comes the check and the printout. */

    val tokens_a_count = tokens.flatMap(t => t.filter(_ == 'a')).count()
    println("OUTPUT: 'a' occurs %d times in tokens".format(tokens_a_count))
    require(!filename.contains("pg2600.txt") || tokens_a_count == 199232)