Useful Unix commands

One of the wonderful things about Unix is that there is a command to do just about anything you could ever conceive of doing. The tricky part is figuring out what the command is called and what switches it takes to do what you want. This page provides a primer on using some of the most important and useful commands to know.

For more information you can use the man command on Unix (e.g. man wc for help on the wc command), although sometimes man pages can be difficult to understand. Another good resource is to search for the command on [http://en.wikipedia.org/wiki/Main_Page Wikipedia].

Navigation

Before you do much of anything else you need to know where you are and how to get somewhere else. Here are some commands to help you get around.

ls

The ls command lists the contents of the current directory. Useful flags include -l to output the listing in long format, which gives more info than just filenames, -h, which in concert with -l shows file sizes in a more human readable format (i.e. bytes, kilobytes, megabytes, etc. instead of disc blocks or byte count), and -a, which shows hidden files. For all of these commands, flags can generally be combined. e.g. ls -lha shows the directory listing in long format with human readable file sizes and includes hidden files.

cd and chdir

cd is short for change directory. (In some shells it is also called chdir.) The directory can be specified as relative to the current directory (e.g. to go to the foo subdirectory of the current directory you'd type cd foo), relative to the root directory of the system (e.g. to go to the bin subdirectory of the usr directory you'd type cd /usr/bin), or relative to your home directory (e.g. to go to the foo subdirectory of your home directory from anywhere on the filesystem you'd type cd ~/foo), or relative to another user's home directory (e.g. to go to the foo subdirectory of a user named bob's home directory you'd type cd ~bob/foo).

There are also a couple of nice shortcuts. If you type cd with no arguments it always takes you back to your home directory. If you type cd - it will take you to the last directory you were in before the current one. This can be useful if you were working in a directory like /Library/Frameworks/Python.framework/Versions/Current/lib/python2.4/, went back to your home directory to do something, and want to get back without typing all that again.

pwd

Probably the most simple command that will be discussed here. All pwd does is print the working directory, i.e. show you where you are in the file system.

File manipulation

glob

Glob is not a command, but it's an important concept for file manipulation. A glob is a way of specifying multiple files that have something in common in their names. There are two globbing characters; the "*", which matches any number of characters and the "?", which matches a single character. e.g. You would type ls *.txt to see a listing of all files in the current directory that have an extension of ".txt".

mv

The mv command moves a directory or file. As arguments it takes the source and the destination. It can also be used to rename a directory or file (e.g. mv foo bar renames the file foo to bar).

cp

The cp command copies a directory or file(s). As arguments it takes the source and the destination. It can take a list of files and will interpret the last argument as the destination and the rest as the source.

rm

The rm command removes (aka deletes) a file. This can be a highly dangerous command, because unlike moving things to the Trash Can on a Mac or the Recycle Bin on Windows, the file is gone as soon you hit enter. Combined with globbing it becomes even more dangerous. Always double check (or triple check) your command line when rming files.

mkdir

The mkdir command makes a directory of the specified name.

rmdir

The rmdir command removes a directory. The directory must be empty to be removed, which can be inconvenient. A much more common way to delete a directory is to use the command rm -rf, which removes files (and directories) recursively through the directory structure and forces the delete if need be. This is possibly the most dangerous Unix command. A misplaced space on the command line and you could delete every file and directory in the current directory. e.g. If you typed rm -rf  foo (note the two spaces before foo) or rm -rf * .txt (note the space between "*" and ".txt"), which would delete all files and then the file called ".txt". If you were logged in as a superuser and accidentally typed rm -rf / foo (again, note the extraneous space) when meaning to delete /foo you would delete all files on the system!

touch

The touch command updates the timestamp on a file to the time that you run the command. If the file doesn't exist, it is created. Usually the command is used to create an empty file of the specified name.

tar

The tar command stands for tape archive. It was originally written to serialize a number of files together into one file for the purpose of putting them on a tape back up that was easy to restore from. These days it's usually used to archive files together similar to a Zip or Rar file. To create a tar file, type tar cvf filename.tar filenames, to unarchive a tar file, type tar xvf filename.tar. Most of the time you'll want to compress the file. In that case use the z option for gzip or the j option for bzip2. (e.g. To create a bzip2'd tar of the directory foo, type tar cvjf foo.tbz foo/.) By convention, the extension for gzipped tar files is .tar.gz or .tgz and the extension for bzip2'd tar files is tar.bz2 or tbz.

Text munging

An overarching theme in the design of Unix is that everything is a stream of characters. Consequently there are a number of commands that allow you to manipulate character streams.

| ("pipe")

Most Unix commands are short, but pipe is just a single character, |, which is usually located on the same key as the backslash. The pipe command allows you to send the output of one command to the input of another. You can string a number of commands this way to make a pipeline where the only the final result is printed.

more and less

The more and less commands output the contents of a file one screenful at a time. more is the older command and can only go forward through a file, i.e. once text has scrolled by you can't get back to it. less is a more sophisticated replacement that allows you to more forward and backward through the file and accepts vi-like commands. Hit h while less is running to see all the commands it accepts. On some systems (e.g. MacOS X) more has been aliased to less.

cat

The cat command is short for concatenate, although in practice that isn't too helpful to know. What it's useful for is outputting the contents of file to the output, which is generally then piped to another command.

If you have a file of compressed text you don't even have to decompress it to cat it, as most systems now have zcat, gzcat, and bzcat, which can cat files compressed with zip, gzip, and bzip2 respectively.

grep

The grep command searches globally for a regular expression and prints matches. The two most common uses for it are to search for files that contain a specific string or match a regular expression, or to do the same for a stream piped into it.

Like cat, grep now has variants that handle compressed files. They are called zipgrep, zgrep, bzgrep for zip, gzip, and bzip2 respectively.

sort

As you would expect, sort takes its input and outputs it in sorted order. Use sort -r to sort in reverse, and sort -f to fold upper and lower case together (meaning that 'A' and 'a' are treated the same; the default is for A-Z to come before a-z).

uniq

The uniq command filters its input such that if multiple consecutive lines are the same it only prints one instance to the output. To be useful the input needs to be sorted already, and as such it is usually used in a pipeline after sort.

wc

wc is short for word count. The default output is the number of lines, words, and bytes in a file. You can use the -l (lines), -w (words), -c (bytes) flags to limit what is shown. You can use the -m flag for number of characters, which is only useful for textfiles with multibyte characters, e.g. Chinese or Japanese.

cut

The cut command is used to cut chunks out of lines in a file. The most common flags to use are -f, which is by field (tab being the default field separator) and -c, which is by character or character range.

paste

The paste command takes an arbitrary number of filenames as arguments and outputs sequentially correspondent lines concatenated together, separated by tabs. In essence, the inverse of cut -f.

sed

sed is short for stream editor. It's actually a full-fledged programming language, but it's often used to write one line scripts to manipulate a character stream. Highly useful to know how to use, but too complex to discuss in detail here.

awk

Like sed above, awk (short for Aho, Weinberger, and Kernighan, last names of its creators) is a full programming language that can also be used inline for character stream manipulation. Again, you should learn it, but it's too complex for this page.