Unix/Linux Command Line Tutorial

© 2019 Brian M. Sandler

If you have a correction or suggestion, please feel free to send it my way.

Introduction

A command line interface is a convenient and powerful way to interact with a computer. It involves some form of a command prompt after which you can enter a command (followed by the return key) for the computer to execute.

Let's open up the "Terminal" program on a Linux or Mac system. (If you do not have a Linux or Mac system, Penn Engineering students can remotely connect to this one here.) We are presented with a prompt like so:

snowbook:~ bms$

On different systems this prompt may vary, but its primary purpose is the same. This prompt is actually just a part of a program called a shell, which is printing the prompt to stdout (e.g. terminal screen by default) and accepting commands via stdin (e.g. keyboard by default). For this tutorial, we're going to use one of the most common shells around, the bash shell. This is almost always what starts automatically when you open a terminal window, and is the default on many Linux distributions (e.g. Ubuntu) and Mac.

When you run commands you are actually running programs (with some exceptions). For example, if you run the command ls to show a directory's contents, what you're actually doing is telling the computer to run a program called ls, and it is that program which outputs content to stdout. Arguments are passed to the program by typing them as space-separated strings after the command (program name). For example:

ls -a

Here ls is the program, and -a is an argument. You can continue to add as many arguments as desired:

somecommand arg1 arg2 arg3 etc...

Navigating the Filesystem

The file system is organized into a hierarchy with the top level folder being /, called the root directory. Every user has a folder (also called a directory) that is their home directory which is meant to store all of their personal files. The path of my home directory is /Users/bms on my Mac. A Users folder sits inside of my root directory (/) and then a folder called bms sits inside of that Users folder.

When a shell is open, it always has a working directory, which defaults to your home directory. Let's see what my working directory is by using the pwd (print working directory) command.

snowbook:~ bms$ pwd
/Users/bms

We can see that it is my home directory. It hasn't changed, and I wouldn't expect it to since I haven't entered any commands yet. There are a handful of commands which will be useful for navigating and manipulating the filesystem:

Let's navigate to our home directory by typing cd. Just typing cd automatically takes you to your home directory. Then let's list its contents with ls.

snowbook:~ bms$ cd
snowbook:~ bms$ ls
Desktop        Music
Documents      Downloads
Movies

Now let's make a folder called tutorial and re-list our home directory.

snowbook:~ bms$ mkdir tutorial
snowbook:~ bms$ ls
Desktop        Music
Documents      Downloads
Movies         tutorial

We can see that this new folder has appeared, now let's "go inside" this folder by changing our working directory to it.

Note: From now on I will be showing the prompt as a "$" for simplicity with notation.

$ cd tutorial

If we type ls we see that it is empty. We can create an empty file using the touch command. Give these commands a try:

$ touch testing.txt
$ ls

The shell can help you type out commands, to show this let's remove the empty file we created. Start typing rm test and then hit the tab key. It should autocomplete the command to rm testing.txt. Once the full command is visible, hit return to execute it. If we ls again our file will be gone.

We can navigate up a directory level by typing cd ... Try out the following commands:

$ pwd
$ cd ..
$ pwd
$ cd tutorial
$ pwd
$ cd ../tutorial
$ pwd

We can use the tilde (~) to specify a path that is relative to our home directory. Try this out:

$ cd ~/tutorial
$ cd ~/
$ mkdir moretutorial
$ cd moretutorial
$ pwd
$ cd ~/tutorial
$ pwd
$ cd ~/moretutorial
$ pwd
$ cd ../tutorial
$ pwd
$ cd ../
$ pwd
$ cd
$ pwd

From your home directory, try to remove moretutorial. 

$ rm moretesting
rm: moretutorial: is a directory

Uh oh, it won't let us! This is because if you want to remove a directory, you must use the -r flag, which means "remove recursively". 

$ rm -r moretutorial

Now let's try to remove the tutorial folder.

$ rm -r tutorial

Let's ls and see that both folders have disappeared. You can then use the clear command to clear your terminal display.

Let's make a folder and file and cd into the folder.

$ cd
$ mkdir tutorial
$ touch ~/tutorial/myfile.txt
$ cd tutorial
$ ls
myfile.txt

Now let's rename myfile.txt to hello.

$ mv myfile.txt hello
$ ls
hello

Sweet! We've renamed it successfully.

Detailed Directory Listing

We can list out more details when we call the ls command by using the -l argument. 

$ ls -l
total 0
-rw-r--r--  1 bms  staff  0 Sep 14 00:06 hello

Here's the breakdown of the important things:

Relative vs. Absolute Paths

You may have noticed that when we use pwd the file path returned always begins with a /. This is because it is an absolute path, a path which starts from the root folder (/). When we cd into a directory we often refer to a folder by its name only, and nothing more. What we're actually doing is specifying a path relative to the current working directory. These kinds of paths are called relative paths. Here are some examples of each.

Relative Paths

Absolute Paths

Hidden Files

Let's go back to our tutorial folder and make a file called .something, and then ls.

$ cd ~/tutorial
$ touch .something
$ ls
hello

Oh no! Our file isn't showing up, this is because files whose names start with a . are considered hidden files. We can make ls show us all the files in our directory by typing:

$ ls -a
.       ..      .something  hello

We now see our hidden file. We also see a reference to the current directory (the .) and the parent directory (the directory one level up, as ..).

Go ahead an remove the tutorial directory.

$ cd ..
$ rm -r tutorial

History

You can see what commands you've previously run by using the history command. 

$ history
  (more commands omitted to be concise)
  544  cd tutorial/
  545  touch hello
  546  ls
  547  touch .something
  548  ls
  549  ls -a
  550  rm .something
  551  ls -l
  552  history

You will see your previously run commands with a number assigned to them. Let's make the tutorial directory again: 

$ cd
$ mkdir tutorial
$ pwd

We can see that we're in the home directory. I want to re-run the cd tutorial/ command, but I don't want to actually type it out!! Instead, I want to re-run it from my history. We can do this by referencing the number next to the command, in my case it is 544. 

$ !544

Now go ahead and type pwd, you'll see that we are now in the tutorial folder. Now I want to re-run the command to create the hello file, in this case it is command 545. 

$ !545
touch hello
$ ls
hello

When we refer to the "!" symbol out loud we call it a "bang".

Another way to quickly scroll through your previously run commands is to use the up arrow key. Try hitting this until you arrive on "pwd" and then press enter.

Go ahead and remove the hello file.

$ rm hello
$ ls

Suppose you don't want to look at a list of old commands and that you know what part of the old command contained. We can perform a "reverse-i-search" by pressing Ctrl-R. Make sure you use the Ctrl key, do not use the command key instead on a Mac. You should get a prompt like so:

(reverse-i-search)`':

Let's type to to find the old touch hello command.

(reverse-i-search)`to': history

Hmm... it seems to want to suggest the history command. I could either continue typing out touch or I can continue pressing Ctrl-R to cycle through the search suggestions. In my case, I now arrive at the command touch hello.

(reverse-i-search)`to': touch hello

Hitting return will execute the command. If I want to cancel the search at any time I can hit the escape key.

The echo command can be used to "echo" what you pass to it as an output to stdout. 

$ echo 'Hello World!'
Hello World!

To re-run the previous command you can type: !! followed by enter.

$ !!
echo 'Hello World!'
Hello World!

Arguments

This section needs improvement.

Optional arguments (i.e. the program can run without them) are sometimes called options. For example, ls -a because ls can run without the -a.

Arguments with a - in front of them but that don't require being followed by an additional value are called flags. Again, ls -a has the flag -a.

Some arguments with a - require parameters which will follow after them. 

$ command -f parameter arg1 arg2 etc...

Getting Help

The man pages (manual pages) are your best friend! Consider them to be documentation for how to use the different commands installed on your system. Let's look at the document for the ls command:

$ man ls

If it prompts you regarding which one you want, just hit return. Your entire window will now be occupied with the man page.

You can see that we have different sections such as "NAME", "SYNOPSIS", and "DESCRIPTION". You can see what the different arguments do and how to use them. The arrow keys work to scroll up and down, and the q key will quit the man page.

To learn more about any command just type man followed by a space and the command name:

man commandname

Some Basic File Tools

Download files from the internet.

Let's cd into the tutorial folder (make one if necessary). From there we'll download the department's homepage as an HTML file.

$ wget -O cis.html https://www.cis.upenn.edu

The cis.html following the -O is an argument for the O (output file) flag. The URL to fetch is the final argument for wget. We can ls to see our newly downloaded file. Let's ls -lh to see its size in human readable form (the h in the argument is for human readable file sizes as opposed to number of bytes, which is the default). 

$ ls -lh
total 72
-rw-r--r--@ 1 bms  staff    34K Sep 14 00:51 cis.html
-rw-r--r--  1 bms  staff     0B Sep 14 00:35 hello

If your system complains about unrecognizing the wget command, you may not have this program installed. You can install it easily using a package manager on Linux (for Mac you can try using Homebrew).

See beginning and end of files.

To show the first three lines of the cis.html file we can use the head command.

$ head -n 3 cis.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml"><!-- InstanceBegin template="/Templates/cis-home.dwt" codeOutsideHTMLIsLocked="true" -->

To see the last three lines:

$ tail -n 3 cis.html

</body>
<!-- InstanceEnd --></html>

In both cases, one line is blank.

Edit a File

Using a text editor we can edit a file. Emacs, vim, and nano are among possible text editors to use. I won't get into how to use them here, but hopefully you know how to use at least one.

Go ahead and edit the file hello to be:

line one
line two
line three

Counting Lines and Words

$ wc hello
       3       6      29 hello

The columns are: new line count, word count, byte count, and file name.

To count just the lines:

$ wc -l hello
       3 hello

All of this information can be found in the man pages (e.g. man wc).

Browsing Large Files

If a file is really large you may not want to open it in your text editor, however you can look at it using the less program. In reality, you can look at a file of any size with less. Try this:

$ less cis.html

Use your arrow keys (or mouse scroll wheel/touchpad) to scroll up and down. Press q to quit, G to go to the end, and g to go to the beginning.

Displaying a file to stdout

The cat command outputs a file to stdout. Think of it as "echo" for files. 

$ cat hello
line one
line two
line three

Searching for lines with a pattern

The grep command let's you find lines in a file that contain a certain pattern. For example:

$ grep 'two' hello
line two

This is a super powerful tool, and you should look more into how to better use it.

You can even search across multiple files with the -r flag!

Piping Commands Together

The commands/programs of the GNU/Linux system are amazing individually, but we can make them even more powerful by piping them together. What this means is we can take the stdout of one program and pipe it into the stdin of another.

For example:

$ cat hello | grep 'two'
line two

The computer will first call cat hello and then pipe it's output (which is the entire hello file) into the stdin of the next program (called by the command grep 'two'). In this case we're able to grep for the lines containing the text "two". This example isn't exciting because we just saw how to use grep by specifying the file to search directly.

Let's modify our text file hello to be:

line one
line two
line three
apple tree
apple cider

Now we're going to count how many lines have the word "apple" in them!

$ cat hello | grep 'apple' | wc -l
       2

Voila! We have two lines which contain the word "apple". There are other ways to do this too, for example we can skip cat.

$ grep 'apple' hello | wc -l
   2

File Permissions

All files and directories on Unix and Linux systems have certain permissions associated with them. You may have gotten errors in the past relating to these permissions, so now it's time to understand the basics about them!

Every file and directory has an owning user and an owning group. Users can belong to many groups, you can see which groups you belong to by typing the groups command. There are three main kinds of permissions you can set on files:

Every file or directory specifies which of these three permissions are granted. A setting for each of these permissions (granted or not) is defined three times on each file. Let's do an ls -lh on a folder on my system.

In my case the permissions are shown as: -rwxr--r--

The first - indicates that this entry is a file and not a directory. If it were a directory this would be a d instead. Next we'll analyze the string.

If you don't have read permission on a file you can't read it or open it. If you don't have write permission you can't modify it or delete it. If you don't have execute permission you can't run it as a program.

Let's look closer at this file's permissions:

-rwxr--r--  1 bms  staff    52B Sep 14 01:37 myprogram

Don't try this right now, but suppose you want to change the owner and/or owning group of a file or folder. Use the chown command: chown bms:www-data myprogram.

Let's play with the permissions of a file. Go ahead and make a new file called test. Let's see what it's default permissions are. For me, they are rw-r--r--. Let's try removing the write permission from the user.

$ chmod u-w test

Now my permissions are: r--r--r--, we can see the write permission is gone from the user. Go ahead and try to modify the file. You won't be able to save your changes. Ok, let's give ourselves the write permission back:

$ chmod u+w test

The arguments for chmod are:

$ chmod [who add/sub permissions] [filename]

Use u for user, g for group, o for other. Use + to add the following permissions or - to remove them. Use r for read, w for write, and x for execute. You can add or remove multiple at once:

$ ls -lh
-r--r--r--  1 bms  staff     0B Sep 14 02:17 test
$ chmod go-r test
$ ls -lh
-r--------  1 bms  staff     0B Sep 14 02:17 test

Similarly you could add read and write to group with g+rw.

Whenever you want to modify permissions I recommend you look at the documentation (the man pages) until you are comfortable. This explanation here is meant to serve as an introduction only.

So what are all those numbers used in chmod commands, like chmod 400 mysecret.key, I've seen around? These numbers are simply another way of expressing read, write, and execute for user, group, and other.

#  Permission              rwx  Binary
--------------------------------------
7  read, write and execute rwx  111
6  read and write          rw-  110
5  read and execute        r-x  101
4  read only               r--  100
3  write and execute       -wx  011
2  write only              -w-  010
1  execute only            --x  001
0  none                    ---  000

Table Source: Wikipedia

With chmod 400 mysecret.key the first digit is for the user (read only), the second for the group (none), and the third for other (none).