CLI apps with Python
Tags: python, beginners, ebook, exercises
Published: 2021-02-03 13:31:12

---
title: "CLI apps with Python"
published: true
description: "Processing CLI arguments using sys and argparse modules in Python"
canonical_url: https://learnbyexample.github.io/100_page_python_intro/command-line-arguments.html
series: 100 Page Python Intro
tags: Python,beginner,ebook,exercises
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/x0gbxspe60srzezgp6gd.png
---

This chapter will show a few examples of processing CLI arguments using `sys` and `argparse` modules. The `fileinput` module is also introduced in this chapter, which is handy for in-place file editing.

## sys.argv

Command line arguments passed when executing a Python program can be accessed as a `list` of strings via `sys.argv`. The first element (index `0`) contains the name of the Python script or `-c` or empty string, depending upon how the Python interpreter was called. Rest of the elements will have the command line arguments, if any were passed along the script to be executed. See [docs.python: sys.argv](https://docs.python.org/3/library/sys.html#sys.argv) for more details.

Here's a program that accepts two numbers passed as CLI arguments and displays the sum only if the input was passed correctly.

```python
# sum_two_nums.py
import ast
import sys

try:
    num1, num2 = sys.argv[1:]
    total = ast.literal_eval(num1) + ast.literal_eval(num2)
except ValueError:
    sys.exit('Error: Please provide exactly two numbers as arguments')
else:
    print(f'{num1} + {num2} = {total}')
```

The [ast.literal_eval()](https://docs.python.org/3/library/ast.html#ast.literal_eval) method is handy for converting a string value to built-in literals, especially for collection data types. If you wanted to use `int()` and `float()` for the above program, you'd have to add logic for separating the input into integers and floating-point first. Passing a string to [sys.exit()](https://docs.python.org/3/library/sys.html#sys.exit) gets printed to the `stderr` stream and sets the exit status as `1` in addition to terminating the script.

Here's a sample run:

```bash
$ python3.9 sum_two_nums.py 2 3.14
2 + 3.14 = 5.140000000000001
$ echo $?
0

$ python3.9 sum_two_nums.py 2 3.14 7
Error: Please provide exactly two numbers as arguments
$ echo $?
1
$ python3.9 sum_two_nums.py 2 abc
Error: Please provide exactly two numbers as arguments
```

As an exercise, modify the above program to handle `TypeError` exceptions. Instead of the output shown below, inform the user about the error using `sys.exit()` method. 

```bash
$ python3.9 sum_two_nums.py 2 [1]
Traceback (most recent call last):
  File "/home/learnbyexample/Python/programs/sum_two_nums.py", line 6, in <module>
    total = ast.literal_eval(num1) + ast.literal_eval(num2)
TypeError: unsupported operand type(s) for +: 'int' and 'list'
```

As another exercise, accept one or more numbers as input arguments. Calculate and display the following details about the input — sum, product and average.

## In-place editing with fileinput

To edit a file in-place, the [fileinput module](https://docs.python.org/3/library/fileinput.html) comes in handy. Here's a program that loops over filenames passed as CLI arguments (i.e. `sys.argv[1:]`), does some processing and writes back the changes to the original input files. You can also provide one or more filenames to the `files` keyword argument, if you do not wish to pass them as CLI arguments.

```python
# inplace_edit.py
import fileinput

with fileinput.input(inplace=True) as f:
    for ip_line in f:
        op_line = ip_line.rstrip('\n').capitalize() + '.'
        print(op_line)
```

Note that unlike `open()`, the `FileInput` object doesn't support `write()` method. However, using `print()` is enough. Here's a sample run:

```bash
$ python3.9 inplace_edit.py [io]p.txt

$ # check if files have changed
$ cat ip.txt
Hi there.
Today is sunny.
Have a nice day.
$ cat op.txt
This is a sample line of text.
Yet another line.

$ # if stdin is passed as input, inplace gets disabled
$ echo 'GooD moRNiNg' | python3.9 inplace_edit.py
Good morning.
```

>As `inplace=True` permanently modifies your input files, it is always a good idea to check your logic on sample files first. That way your data wouldn't be lost because of an error in your program. You can also ask `fileinput` to create backups if you need to recover original files later — for example, `backup='.bkp'` will create backups by adding `.bkp` as the suffix to the original filenames.

## argparse

`sys.argv` is good enough for simple use cases. If you wish to create a CLI application with various kinds of flags and arguments (some of which may be optional/mandatory) and so on, use a module such as the built-in `argparse` or a third-party solution like [click](https://pypi.org/project/click/).

Quoting from [docs.python: argparse](https://docs.python.org/3/library/argparse.html):

>The `argparse` module makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and `argparse` will figure out how to parse those out of `sys.argv`. The `argparse` module also automatically generates help and usage messages and issues errors when users give the program invalid arguments.

Here's a CLI application that accepts a file containing a list of filenames that are to be sorted by their extension. Files with the same extension are further sorted in ascending order. The program also implements an optional flag to remove duplicate entries.

```python
# sort_ext.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-f', '--file', required=True,
                    help="input file to be sorted")
parser.add_argument('-u', '--unique', action='store_true',
                    help="sort uniquely")
args = parser.parse_args()

ip_lines = open(args.file).readlines()
if args.unique:
    ip_lines = set(ip_lines)

op_lines = sorted(ip_lines, key=lambda s: (s.rsplit('.', 1)[-1], s))
for line in op_lines:
    print(line, end='')
```

The documentation for the CLI application is generated automatically based on the information passed to the parser. You can use help options (which is added automatically too) to view the documentation, as shown below:

```bash
$ python3.9 sort_ext.py -h
usage: sort_ext.py [-h] -f FILE [-u]

optional arguments:
  -h, --help            show this help message and exit
  -f FILE, --file FILE  input file to be sorted
  -u, --unique          sort uniquely

$ python3.9 sort_ext.py
usage: sort_ext.py [-h] -f FILE [-u]
sort_ext.py: error: the following arguments are required: -f/--file
```

The `add_argument()` method allows you to add details about an option/argument for the CLI application. The first parameter names an argument or option (starts with `-`). The `help` keyword argument lets you add documentation for that particular option/argument. See [docs.python: add_argument](https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_argument) for documentation and details about other keyword arguments.

The above program adds two options, one to store the filename to be sorted and the other to act as a flag for sorting uniquely. Here's a sample text file that needs to be sorted based on the extension.

```bash
$ cat sample.txt
input.log
basic.test
input.log
out.put.txt
sync.py
input.log
async.txt
```

Here's the output with both types of sorting supported by the program.

```bash
# default sort
$ python3.9 sort_ext.py -f sample.txt
input.log
input.log
input.log
sync.py
basic.test
async.txt
out.put.txt

# unique sort
$ python3.9 sort_ext.py -uf sample.txt
input.log
sync.py
basic.test
async.txt
out.put.txt
```

>See [docs.python HOWTOs: Argparse Tutorial](https://docs.python.org/3/howto/argparse.html) for a more detailed introduction.

## Accepting stdin

CLI tools like `grep`, `sed`, `awk` and many others can accept data from `stdin` as well as accept filenames as arguments. The previous program modified to add `stdin` functionality is shown below. `args.file` is now a positional argument instead of an option. `nargs='?'` indicates that this argument is optional. `type=argparse.FileType('r')` allows you to automatically get a filehandle in `read` mode for the filename supplied as an argument. If filename isn't provided, `default=sys.stdin` kicks in and you get a filehandle for the `stdin` data.

```python
# sort_ext_stdin.py
import argparse, sys

parser = argparse.ArgumentParser()
parser.add_argument('file', nargs='?',
                    type=argparse.FileType('r'), default=sys.stdin,
                    help="input file to be sorted")
parser.add_argument('-u', '--unique', action='store_true',
                    help="sort uniquely")
args = parser.parse_args()

ip_lines = args.file.readlines()
if args.unique:
    ip_lines = set(ip_lines)

op_lines = sorted(ip_lines, key=lambda s: (s.rsplit('.', 1)[-1], s))
for line in op_lines:
    print(line, end='')
```

Here's the help for the modified program:

```bash
$ python3.9 sort_ext_stdin.py -h
usage: sort_ext_stdin.py [-h] [-u] [file]

positional arguments:
  file          input file to be sorted

optional arguments:
  -h, --help    show this help message and exit
  -u, --unique  sort uniquely
```

Here's a sample run showing both `stdin` and filename argument functionality.

```bash
# 'cat' is used here for illustration purposes only
$ cat sample.txt | python3.9 sort_ext_stdin.py
input.log
input.log
input.log
sync.py
basic.test
async.txt
out.put.txt
$ python3.9 sort_ext_stdin.py -u sample.txt
input.log
sync.py
basic.test
async.txt
out.put.txt
```

As an exercise, add `-o, --output` optional argument to store the output in a file for the above program.