CSC 161 Grinnell College Spring, 2010
 
Imperative Problem Solving and Data Structures
 

Linux and C

Goal

This lab continues our exploration of Linux, including pipes, filters, additional Linux utilities, wildcards, quoting, and command substitution.

Prerequisites

This discussion builds on the Linux commands and framework included in the introductory Linux lab and the discussion of I/O redirection in the lab on testing.

Pipes and Filters

One powerful type of I/O redirection within Linux is called a "pipe" (represented by the vertical bar "|", which on many keyboards is located on the same key as the backslash). A pipe allows you to connect programs in a "pipeline." A pipe causes the data sent to stdout from one program to be re-directed into stdin of another program. We say that the output from the first program is "piped" to the input of the next program.

For example, the following command allows you to page through a long directory listing.

   ls -l /bin | more

In this line, the initial command ls -l /bin creates a detailed directory listing, and sends the information to stdout — by default, the terminal. The pipe | channels this output into the input of the more programs for display.

The fact that many Linux programs can accept their input from one file, multiple files, or stdin makes them very versatile. Pipes take this versatility one step farther: any command or program that can accept input from stdin can also accept input from a pipe. Any command or program that sends output to stdout can also send output through a pipe.

To combine more than one program, we can string multiple pipes together to create longer pipelines, like so:

   command | command | command

Additional Linux Utilities

Many Linux utilities that accept input from stdin and send output to stdout are known as "filters" because, in one way or another, they filter their input to produce output. These utilities are especially apt for creating useful pipelines. In the first laboratory on Basic Linux Commands, you have worked with several filters, including less, tail, and cat.

The following table lists some additional filters. All of these filters can accept their input from regular files, stdin, or a pipe. None of them modify their original input; rather, they generate new output that reflects a modified version of the input.

Utility Description Example usage
wc "word count" - counts characters, words, and lines in input wc -l ~/.bashrc
sort sorts lines in input sort -k2 ~coahranm/share/csc201/sciencefac.txt
uniq "unique" - removes (or reports) duplicate lines in input uniq ~coahranm/share/csc201/duplicates.txt
grep searches for a target string in input grep li ~coahranm/share/csc201/duplicates.txt
cut removes parts of lines from input cut -d' ' -f2 ~coahranm/share/csc201/sciencefac.txt
diff reports the differences between two files diff ~walker/public_html/courses/153.sp09/labs/lab-linux-c.shtml ~walker/public_html/courses/153.sp09/labs/lab-linux-c.shtml~

A Few More Useful Linux Capabilities

To conclude this lab, we briefly describe a few useful capabilities. In each case, these options can be worthwhile as you work within the Linux environment, but in the interests of time we provide few experiments here.

Wildcards

You are probably already familiar with the idea of "wildcards" in filenames. For example, you can use the command "ls *.ss" to get a listing of all the files with the extension .ss.

What allows this to work? The shell parses your input, discovers the asterisk in it, and "expands" the command to include all files that match the given pattern. This ability to expand commands based on special characters in the input is also called globbing.

Further, the asterisk is not the only special character used for globbing. Here are some more.

Special Character Is replaced by... Example(s)
* matches any string (including zero characters) cat ~coahranm/share/csc201/*fac.txt | less
? matches any single character ls -ld /usr/bin/gc?
ls /usr/lib/lib?.a
[...] matches any single character inside the brackets ls /usr/lib/lib[xX]*.a

Quoting

On occasion, you want to keep the shell from treating special characters specially. For example, suppose you receive the file, "class pictures", from your friends. To list the file in a terminal window, you might write

   cat class pictures

However, Linux will interpret this command as a request to list file "class" and then file "pictures". To resolve this confusion, you can proceed in either of two ways.

  1. We can quote the file name, as follows. The single quotation marks keep the shell from treating characters inside them specially. In this case, it keeps the shell from parsing the line into three separate tokens.

       cat 'class pictures'
    
  2. We can escape individual characters with a backslash. Again this keeps the shell from interpreting these characters in their usual (specialized) way. Be sure to try this one as well.

       cat class\ pictures
    

Command Substitution

Command substitution allows you to embed one command inside another, using "backquotes" to delimit the nested command. (You should be able to find the backquote character in the upper-left of the keyboard, with the tilde.)

When you do this, the shell first executes the backquoted command, then substitutes the result that the command output to stdout in place of the command itself. Finally, the shell interprets and runs the resulting command string.

For example, consider the command

   date +%A

This prints the day of the week (e.g., Sunday or Monday). To incorporate this result into an echo statement, you might write:

  echo Today is `date +%A`.

When we begin writing shell scripts later in the semester, you will find this ability useful for creating informative output messages.


This document is available on the World Wide Web as

     http://www.cs.grinnell.edu/~walker/courses/161.sp10/labs/lab-linux-c.html

created January 2007 by Marge Coahran
revised January 2008 by Marge Coahran
revised 10 April 2008 by Henry M. Walker
revised 3 April 2009 by Henry M. Walker
reorganized 2 April 2010 by Henry M. Walker
last revised 3 April 2010 by Henry M. Walker
Valid HTML 4.01! Valid CSS!
For more information, please contact Henry M. Walker at walker@cs.grinnell.edu.