# LPI Linux Certification/LPIC1 Exam 101/GNU & Unix Commands

* LPIC1 Exam 101
* Detailed_Objectives
* Hardware & Architecture
* Linux Installation & Package Management
* GNU & Unix Commands
* Linux Filesystems
* The X Window System
* Exercises results

## Section Overview

* 1.103.1 Work on the command line
* 1.103.2 Process text streams using filters
* 1.103.3 Perform basic file management
* 1.103.4 Use streams, pipes, and redirects
* 1.103.5 Create, monitor, and kill processes
* 1.103.6 Modify process execution priorities
* 1.103.7 Search text files using regular expressions
* 1.103.8 Perform basic file editing operations using VI
* Unix and Linux

## Unix and Linux
### Overview

Unix is the original Internet Operating System (NFS, TCP/IP, RPC, ...). Linux is a Unix clone written from scratch by Linus Torvalds with assistance from a world wide team of developers across the Net. Properties of both systems are:
* Portable, multi-user, multiprocessing
* Device-independent input/output
* Shared resources, high performance and highly secure
* Line commands
* GUI
Linux aims toward POSIX compliance and supports also other standard API's such as BSD and SVR4. Linux has all the features you would expect in a modern Unix, including true multitasking/multiple-users, virtual memory, shared libraries, demand loading, shared copy-on-write executables, proper memory management and TCP/IP networking.
More and more companies and organizations are selecting Linux because of its performance, low cost, and royalty free license.

#### Unix and Linux

Unix is the original Internet Operating System written in 1970 developed at A&T Bell Labs. Distributed freely to government and universities. Different entities did support different distributions.
* System V
* BSD
* SCO
Most varieties of Unix carry copyright licenses.
Linux was released in 1991 by Linus Torvalds. The model of Linux licence is copy left from GPL. Linux implement most of the System V and BSD Unix commands. Use the man pages to determine the version.

#### Logging In and Out

The logging In session is the first step to access a Linux system. You need a login ID and a password to be able to start using the system. The login ID root is the super user ID that has all privileges, normally assigned to the System Administrator. When a log in session is successful it will set some environment variables and start the shell that has been assigned to the login ID account.
Examples of variables set at log in time: $HOME, $SHELL, $PATH, and more. By changing the file /etc/login.defs the administrator can customize some login parameters and variables for all the users.
Examples of some of them:
To logout use the exit or logout command.

#### Password

Nobody can see your password, not even the administrator. To change passwords for users or groups, use passwd.
Common options:
* -l, -u : Lock and unlock an account.
* -g [| r]: Used for group.
* -g set a password for group and -r removes it.
* -x|n : Maximum and minimum number of days a password remain valid.
As a general rule a password must:
* Have a minimum length
* Not contain any English language words
* Contain non-alphanumeric characters
* Contain a mixture of uppercase and lower case letters.

#### User information

For every user account, there is a line defined in the file /etc/passwd. The encrypted password is stored in /etc/shadow. The format of /etc/passwd is:
The x field is the old location for the password.
The format of /etc/shadow is:
The * for the encrypted password means there is no password defined yet. The ! Before the encrypted password means the account is locked.

#### Group information

For every group of users there is a line in file /etc/group. The encrypted password is stored in /etc/gshadow. The format of /etc/group is:
The format of /etc/gshadow is:
Passwords for group are not often implemented since you have to pass to members of a group the password.

#### Misc. Commands

To get information on a logging In session, use id.
Common options: -g : Print only the group ID. -u : Print only the user ID.
Example:
To run a shell with another user ID and group ID, use su.
Common options: -s : Select another shell.
Example:

#### Shell

A Shell is a command line interpreter and:
* Prompts for commands.
* Interprets command which can include:
* I/O redirection, pipes, filename generation, etc.
* Executes commands.
* Executes scripting language.
There are several popular shells:
* sh: Bourne shell, the standard scripting shell.
* csh: Berkeley shell, job control, command history, alias.
* ksh: Korn shell, Bourne syntax with Berkeley features.
* bash: Bourne Again shell, free, Korn-like shell.
* t csh: Free, Berkeley clone with many extensions.
Most shells let you customize your prompt and environment. For bash the prompt variable is PS1. For bash the file that can be used for the customization of the environment are:
* .bashrc Executed for each new bash session
* .bash_profile Executed at logging In time
* .bash_logout Executed at logging Out time

### Exercises

* Exercises results

## Process text streams using filters
### Overview

Description: Candidates should should be able to apply filters to text streams. Tasks include sending text files and output streams through text utility filters to modify the output, and using standard UNIX commands found in the GNU textutils package.
Key files terms and utilities include: cat cut expand fmt head join nl od paste pr sed sort split tac tail tr unexpand uniq wc

### Pattern matching and wildcards

Wildcard is a pattern matching mechanism for filenames. The purpose of wildcard is to increase productivity: Locate files that you don't fully remember. Locate files that have something in common. Work with a group of files rather than individual.
The shell interprets these special characters:
The characters used for wildcard are:
If you use the wildcard characters the shell will try to generate a file from them. Try the following:
Special wildcard characters
Examples:
What about these commands?

### Shell and wildcards

A shell command line can be a simple command or more complex.
The first event in the shell is to interpret wildcards. Only the shell interprets unquoted wildcards.

### Quoting and Comments
#### Quoting

Do quote to prevent the shell to interpret the special characters and to transform multiple words into one shell word.
* 'string' Nearly everything within the quote is literal:
* "string" Like 'string' but interprets $, \, !:
* The backslash \ treats the following character as literal:
* How could we display the backslash? With the following line:

#### Comments

You can add comments in a command line or a script. Use the character #. A white space must immediately precede #.
Examples:
Commands
* cat, tac: Concatenate files and print on the standard output.
* head, tail: Output the first and last part of files.
* nl: Number lines of files.
* wc: Print the number of bytes, words, and lines in files.
* cut: Remove sections from each line of files.
* tr: Translate or delete character.
* expand, unexpand: Convert tabs to spaces and space to tabs.
* paste: Merge lines of files.
* join: Join lines of two files on a common field.
* uniq: Remove duplicate lines from a sorted file.
* split: Split a file into pieces.
* fmt: Simple optimal text formatter.
* pr: Convert text files for printing.
* sort: Sort lines of text files.
* od: Dump files in octal and other formats.

### Concatenate files

To concatenate files, use cat.
The results are displayed to the standard output.
Common options:
Examples:
To concatenate files in reverse order, use tac.

### View the beginning and the end of a file

To view only few lines at the beginning or at the end of a file, use head or tail.
The results are displayed to the standard output.
Common options:
Examples:

### Numbering file lines

To add the line number to a file, use nl.
The results are displayed to the standard output.
Common options:
Examples:

### Counting items in a file

To print the number of lines, words and characters of a file, use wc.
The results are displayed to the standard output.
Common options:
Examples:

### Cutting fields in files

To remove sections from each line of files, use cut.
The results are displayed to the standard output.
Common options:
Examples:
The default delimiter is TAB but can be specified with -d.

### Characters conversion

To translate the stdin to stdout, use tr.
Common options: -d: delete character in SET1. -s: replace sequence of character in SET1 by one.
Examples:
To convert tabs to spaces, use expand and to convert spaces to tabs, use unexpand.

### Lines manipulation

To paste multiple lines of files, use paste.
Common options:
Examples:
To join multiple lines of files, use join.
To remove duplicated line, use uniq.
Common options:
Examples:
Splitting files
To split big files, use split.
Common options:
Examples:

### Formatting for printing

To format a file, use fmt.
Common options: -w #: maximum line width.
Examples:
To format a file for a printer, use pr.
Common options: -d: double space.
Examples:

### Sort lines of text files

To sort the lines of the names files, use sort.
sort [options] file
The results are displayed to the standard output.
Common options:
Examples:

### Binary file dump

To dump a binary file, use od.
The results are displayed to the standard output and start with an offset address in octal.
Common options:
Examples:

### Exercises

* Use wildcard characters and list all filenames that contain 'x' followed by 'in' in the /etc directory.
* Use wildcard characters and list all filenames that start with any character between 'a' and 'e' that have at least two more characters and do not end with a number.
* Use wildcard characters and list all filenames of exactly 4 characters and all filenames starting with an uppercase letter. Do not descend into any directory found.
* Use wildcard characters and list all files that contains 'sh' in /bin.
* Display you environment variable HOME preceded by the string "$HOME value is:"
* Display the contents of $SHELL with two asterisk characters before and after it.
* How would you display the following string of characters as is with echo using double quote and \.
* Compose echo commands to display the following two strings:
** That's what he said!
** 'Never Again!' he replied.
* Display the number of words in all files that begin with the letter h in the /etc directory.
* How would you send a 2M file with two 1.44M floppy. How would you put back together the pieces of the split file?
* What is the command to translate the : delimiter in /etc/password by #?
* Exercises results

## Perform basic file management
### Overview

Description: Candidates should be able to use the basic UNIX commands to copy, move, and remove files and directories. Tasks include advanced file management operations such as copying multiple files recursively, removing directories recursively, and moving files that meet a wildcard pattern. This includes using simple and advanced wildcard specifications to refer to files, as well as using find to locate and act on files based on type, size, or time.
Key files terms and utilities include: cp find mkdir mv ls rm rmdir touch file globbing
cp: Copy files and directories. ls: List directory contents.

### Create and Remove directories

To create a directory, use mkdir.
Common options:
Examples:
To delete an empty directory, use rmdir.
Common options:
Examples:

### Copy files and directories

To copy one file to another, or to a directory, use cp.
Source and target can be a file or a directory.
Common options:
The default is to silently clobber the target file. Does not alter the source.
Examples:

### Move & Rename files

To rename a file or directory or to move a file or directory to another location, use mv.
Source and target can be a file or a directory.
Common options:
The default is to silently clobber the target file.
Examples:

### Listing filenames and information

The command to list files in the current directory is ls.
Common options are:
The ls is equivalent to the dir command on DOS.
Examples of ls output:

### File types

The long format means:
File content and location Linux/Unix does not distinguish file by filename extension, like Windows. To determine the file content use file.
To determine if a command is a built-in shell command or a program, use type, and use which to find its location.

### Creating and using filenames

Filenames can be created with:
* I/O redirection
* An editor, such as vi.
* Many of the Unix utilities
* An application
* The touch command, which creates empty files (or updates the "date modified" of existing files)
The valid filename may have:
* Maximum 255 characters per filename
* Case sensitive 'A' and 'a' are different
* Any character except forward '/'
* Recommended alphanumeric characters as well as plus, minus, and underscore characters.
* Characters to avoid
* Hypen character.
* touch my-file -lt
* White space.
* Most other special characters !@#$%^&*:;"'}{|\<,>.?~`

### Remove files or directories

To remove files or subtree directories, use rm.
Files can be a file or a directory.
Common options:
There is no unremove or undelete command.
Examples:

### Locating files in a subtree directory

To search for a file in a subtree directory, use find.
The command can take multiple conditions and will search recursively in the subtree.
Some possible conditions are:
Some possible actions are:
Examples
To locate a binary, source file, or man pages, use whereis.
Common options:
Examples:
To locate a file located somewhere defined by the PATH variable, use which.
The -a will look for all possible matches in PATH, not just for the first one.

### Exercises

* Compose an interactive command to remove all .tmp file in your home directory. Respond y to every prompt.
* List all the files in users home directories ending with .pdf that are bigger than 50 blocks and have not been accessed for a month.
* Create a file file.h that will contain all the filenames ending with .h found in the /usr directory.
* Do a touch on all the c files found in /usr/src/packages directory.
* What are the default permissions when you create a new file and a new directory?
* How would you create a new file or directory that contains a space in the filename? (Example: 'new dir')
* What is the command to remove all the files of type char and block in your home directory?
* How to find the location of the programm find ?
* Delete all files in /tmp witch are not owned by root and have not been accessed for a week.
* Exercises results.

## Use streams, pipes, and redirects
### Overview

Description: Candidates should be able to redirect streams and connect them in order to efficiently process textual data. Tasks include redirecting standard input, standard output, and standard error, piping the output of one command to the input of another command, using the output of one command as arguments to another command and sending output to both stdout and a file.
Key files terms and utilities include: tee xargs < << > >> | ` `

### Standard input and standard output

For each command executed in a terminal, there are: a standard input value 0. (default keyboard) a standard output value 1.(default terminal) and a standard output for the errors value 2. (default terminal).
Each channel can also be identified by an address: &0 for input, &1 for output, And &2 for errors.
Each channel [n] can be redirected. [n]< file: Default value of n is 0 and it reads standard input from file. [n]> file: Default value is 1 and it sends standard output to file, overwriting the file if it exists. (clobber) [n]>>file: Default value is 1 and it appends standard output to file. <<word: Read standard input until word is reached. `command`: Substitute the command name by the result.
Examples:
Concatenate all the configuration files from /etc dir in conffile and append errors in file /tmp/errors.
Redirecting with pipes
Pipes are an efficient way to apply multiple commandes concurrently.
The standard output of command1 will be piped to the standard input of command2. The standard error is not piped.
Examples:
To redirect the standard output to a file and to the terminal at the same time, use tee.
Building arguments
The xargs utility constructs an argument list for a command using standard input.
The xargs command creates an argument list for command from standard input. It is typically used with a pipe.
Common options: -p: prompt the user before executing each command.
Examples:
Search in the Home directory for files starting with proj1 and send it to the standard input of cat.
Use the /dev/null device file to discard output or error messages.
Try the following:

### Exercises

* Create a file list.bin that will contain all the filenames from /bin directory.
* Write a command that will append the list of files from /usr/local/bin to the file named list.bin and discard any error output.
* Split list.bin file into files that are 50 lines long and remove list.bin.
* From the splitted files create back list.bin. but with inversed order.
* Simplify the following commands:
* Use find to do the following command:
* Write a command that will create a file list.sbin with the contents of /sbin and at the same time display it to standard output.
* Create a file that within the filename you include the creation time.
* Create a file that will contain all the filename in reverse order with extension .conf from the /etc directory.
* Exercises results.

## Create, monitor, and kill processes
### Overview

Description: Candidates should be able to manage processes. This includes knowing how to run jobs in the foreground and background, bring a job from the background to the foreground and vice versa, start a process that will run without being connected to a terminal and signal a program to continue running after logout. Tasks also include monitoring active processes, selecting and sorting processes for display, sending signals to processes, killing processes and identifying and killing X applications that did not terminate after the X session closed.
Key files terms and utilities include: & bg fg jobs kill nohup ps top

### Create processes

A running application is a process. Every processes have: A process ID. A parent process ID. A current directory PWD. A file descriptor table. A program which it is executing. Environment variables, inherited from its parent process. Stdin, stdou, and stderr Other
Bash is a program that when it is executed becomes a process. Each time you execute a command in a shell a new process is created. Except for the buil-in shell command. They run in the shell context. Use type to check if a command is a built-in shell command.
Example type cp ls which type Monitor processes Once the system is up and running from a terminal it is possible to see which processes are running with the ps program. To display a long format of all the processes in the system, do:
The ps program will display all the processes running and their PID numbers and other information. To see a long format of the processes in your login session, do a:

### Monitor processes

To monitor the processes in real-time, use top. top
78 processes: 75 sleeping, 3 running, 0 zombie, 0 stopped CPU states: 15.3% user, 0.3% system, 0.0% nice, 84.2% idle Mem: 254896K av, 251204K used, 3692K free, 0K shrd, 27384K buff Swap: 514072K av, 0K used, 514072K free 120488K cached
RSS: The total amount of physical memory used by the task. SHARE: The amount of shared memory used by the task. %CPU: The task's share of the CPU time. %MEM: The task's share of the physical memory. Once top is running it is also possible to execute interactive commands: Type N to sort tasks by pid. Type A to sort tasks by age (newest first). Type P to sort tasks by CPU usage. Type M to sort tasks by memory usage. Type k to kill a process (prompted for pid).

### Kill processes

The ps program will display all the processes running and their PID numbers. Once the PID is known, it is possible to send signals to the process. SIGSTOP to stop a process. SIGCONT to continue a stopped process. SIGKILL to kill a process.
The program to send a signal to a process is called kill.
By default a process is started in foreground and it is the only one to receive keyboard input. Use CTRL+Z to suspend it.
To start a process in background use the &.
Job control
In a bash process it is possible to start multiple jobs. The command to manipulate jobs are:
When bash is terminated all processes that have been started from the session will receive the SIGHUP signal. This will by default terminate the process.
To prevent the termination of a process, the program can be started with the nohup command.

### Exercises

* Exercises results.

## Modify process execution priorities
### Overview

Description: Candidates should should be able to manage process execution priorities. Tasks include running a program with higher or lower priority, determining the priority of a process and changing the priority of a running process.
Key files terms and utilities include: nice ps renice top

### Priorities

To start a command with an adjusted priority, use nice. nice -n +2 [command] nice -n -19 [command] The program nice changes the base time quantum of the scheduler. i.e. it informs the scheduler how important a process is, which is used as a guide to how much CPU time to give it.
For example if you wanted to listen to music whilst ripping another CD, you could use nice -n +5 oggenc, you shouldn’t get any “hops” in the music playback, as the scheduler “knows” the oggenc process is less important.
The value can go from -19 (highest priority) to +20 (lowest priority). The default value is 0. Only root can set a value below zero.
To modify the priority of a running program, use renice. renice +1 -u root # Change the priority for all root processes. renice +2 -p 193 # Change the priority for PID 193

### Exercises

* Exercises results
* Which user and root processes are using most of the memory?
* Open two terminals. In one terminal type the following:
From the other terminal see that you can stop and continue the print out.
3) Same start as 2) but, make the print out to stop for 3[s] and to continue for 1[s] repeatedly. 4) Make a shell script to renice all process called apache to a 19 value. 5) Do a print from ps formated as: “username”, “command”, “nice value” 6) Kill all the process called “bash” and owned by user polto..

## Regular Expression
### Overview

Description: Candidates should be able to manipulate files and text data using regular expressions. This objective includes creating simple regular expressions containing several notational elements. It also includes using regular expression tools to perform searches through a filesystem or file content.
Key files terms and utilities include: grep regexp sed

### Pattern matching

There are two kinds of pattern matching:
* Wildcards (File Name Generation)
* Regexp (Regular Expression)
Wildcard characters are mainly applied when they are used in the current directory or subdirectories. When wildcard characters *, ?, [ - ], ~, and ! are used in regexp they no longer generate filenames.
Some of the utilities that use regexp are:
* grep, egrep
* vi
* more
* sed
* Perl
Limited regexp search pattern Used by all utilities using regexp.
* Any 1 char . Ab.a Abla or Abca
* 1 char set [ ] Ab[sd]a Absa or Abda only
* 1 char range [ - ] Ab[a-z]a Abaa or Abba or ...
* Not in set [^ ] Ab[^0-9]a Abaa or Abba or ...
* 0 or more * Ab*a Absala or Aba or ...
* Begin line ^ ^Aba Line starts>Aba
* End line $ Aba$ Aba<line ends
* Literal \ Aba\$ Aba$
Example:
Combinations of limited regexp Combinations used by all utilities using regexp.
* Any string .* Ab.*a Abrahma or Abaa or ...
* String from [ ]* th[aersti]* There or This or ...
* Multi range [ - - ] Ab[0-2][a-c]a Ab0aa or Ab1aa or ...
* Match \ \\ \\[a-zA-Z]* \Bethoven
Examples:
Modifier patterns Replace strings matched by regexp patterns
* Match m \{m\} b[0-9]\{3\} b911
* One or more \{m,\} b[0-9]\{2,\} b52
* Up to n \{m,n\} b[0-9]\{2,4\} b1234
* Beginning of word \< \<wh where
* End of word \> [0-9]\> bin01

### grep

To find text in a file, use grep.
Best to quote the string to prevent interpretation.
Common options: -i: Ignore case -l: List filename only if at least one matches -c: Display only count of matched lines -n: Also display line number -v: Must not match.
Examples:

### sed

To apply a command on a stream, use sed.
The program sed will apply a command from address1 to address2 in a file. The address1 and address2 format are regular expressions. The sed program is a noninteractive editing tool.
Examples:

### Exercises

* Exercises results
* Process your bookmarks.html file to produces a list containing just the web site titles in a file called mywebsites.txt
* Copy all the files from /etc in your home direcory etc/. Display the contents of all the *.conf files by replacing the word host by machine.
* Display the contents of all the *.conf files that don't contain the root word. What is the command using grep and sed?
* Print out all the group names where root belongs to.
* List all the group names that are 4 or 5 characters long.
* List all the files that contain character lines without spaces (blank).
* List in etc/ directory all the files that contain numerical characters.
* Print with ls only the directory names in /.
* Do “ps -aux” and replace user r_polto by root and print it in to a file called new_process.txt
* List all process called apache and owned by a usernames starting by “p” or “P”..

## Perform basic file editing using vi
### Overview

Description: Candidates should be able to edit text files using vi. This objective includes vi navigation, basic vi nodes, inserting, editing, deleting, copying, and finding text.
Key files terms and utilities include: vi /, ? h,j,k,l G, H, L i, c, d, dd, p, o, a ZZ, :w!, :q!, :e! :!

### vi

When using X-Windows, you can use mouse-oriented editors such as xedit. In a cross-development environment, users use their favorite editor. On a non-windowing system, you only need a keyboard editor such as vi. The vi editor on Linux is the same as on any Unix systems. vi has two modes:
* Command mode: Anything you type will be interpreted as a command
* Input mode: Anything you type will be inserted into the file
Transition from one mode to another Command mode to Input mode: i, I, a, A, o, O keys Input mode to Command mode: ESC key The default starting mode is the Command mode The file configuration .exrc can be created in your HOME directory to set up some vi behavior.
Perform basic file editing using vi
Enter Input mode
* i before cursor
* I at start of line
* a after cursor
* A at end of line
* o open line below
* O open line above
Delete
* dw delete word
* dd delete line
* D delete to end of line
* x delete char at cursor
Move cursor
* l one space right
* h one space left
* j one line down
* k one line up
* $ end of line
* ^ start of line
* w next word
* e end of word

### Exercices

* Use vi from any directory to begin editing an empty buffer.
* Enter a few lines of text into this buffer.
* Save the contents of the buffer to the directory.
* Open the file again with vi.
* Create a new line beneath what you typed earlier. (Don't use i from command mode!)
* Exit vi without saving these changes.
* Exercises results