?

Log in

 
 
10 January 2009 @ 05:45 pm
notes: Perl Commandline Primer  
Hey, I found some really old notes that I wrote, which are still mostly relevant today! This is a Primer for Perl on the Command Line. It's been available at http://impson.tzo.com/~jdimpson/proj/perl_commandline_primer.txt since I wrote it in 1998. I wrote it for a Linux User Group that I helped start around that time (http://www.stnylug.org/). I've always loved Perl for its flexibility, and using it on the command line (vs writing out perl scripts) for one-shot administrative tasks is for me a big part of that flexibility. Here's the primer in its entirety, unmodified from how it was originally presented at an early LUG meeting. Apparantly, it was presented January 19, 1999. Hmm. According to earlier meeting minutes, the notion of presenting these notes was conceived in early 1999, not in 1998, so that's more likely when this was written, despite the copyright date in the file.

BEGIN PRIMER

These notes are Copyright 1998 by Jeremy D. Impson <jdimpson@acm.org>.
You may copy them in their entirety or in part for your own use (any use)
provided this copyright and credit notice is included in the copy.

----------------
- Running perl -
----------------

1. Perl on the command line is an incredible administrative tool and a great time server

2. Normal way to run a perl script is by writing the code into file (say, "foo") then
running it like this

$ cat foo
print "hello, world!\n";

$ perl foo
hello, world!


3. If the script has "#!/usr/bin/perl" as the first line, then you can run the script
directly as an application:

$ cat foo
#!/usr/bin/perl
print "hello, world!\n";

$ foo
hello, world!

4. Sometimes writing files out is a pain, especially for one shot deals or when you don't
have write permission. The same affects as above can be gained with the '-e' flag, which says
"execute the next chunk as Perl code".

$ perl -e 'print "hello, world\n";'
hello, world!

------------------
- Quotation tips -
------------------

5. Must group the argument to -e with either single or double quotes so that the shell will
pass the whole thing as the first argument. Single quotes are recommended, because they prevent
the shell from trying to interpolate various characters that are special to both Perl and the
shell.

To use a single quote within your Perl code, use the "q/string/" notation, which perl
will treat just like single quotes.

$ perl -e 'print q/hello, world\n/;'
hello, world\n

(Single quotes (or q//) prevent \n from being interpretted as a newline.)

Another way to write double quotes is to use the "qq./string/" notation, which perl
will treat just like double quotes.

So

$ perl -e 'print qq/hello, world\n/;'
hello, world

works _exactly_ like

$ perl -e 'print "hello, world\n";'
hello, world

---------------------
- Commandline flags -
---------------------

6. '-e' can be used multiple times. Each script that follows each '-e' will be run
as though it were the next line of the script.

$ perl -e 'print "1\n"; die;' -e 'print "2\n";'
1
Died at -e line 1.

$ perl -e 'print "1\n"; ' -e 'print "2\n";die;'
1
2
Died at -e line 2.

7. '-c' can be used to check the syntax of a file without executing it, to see if there are
any syntactical bugs.

$ cat foo
prunt "hello, world\n";

$ perl -c foo
String found where operator expected at foo line 1, near "prunt "hello, world\n""
(Do you need to predeclare prunt?)
syntax error at foo line 1, near "prunt "hello, world\n""
foo had compilation errors.

(Basically, it is saying that it doesn't know what "prunt" is.)

8. '-n' and '-p' both implicitly add a wrapper around your code. '-n' adds this:

foreach (<>) {

# your code goes here

}

and '-p' adds this:

foreach (<>) {

# your code goes here

print $_;
}

The effect is that first, all data read in from standard input will have your code applied
to it, then second, each line of each file listed on the commandline will have your code
applied to it. For the case of '-p', after your code applied, the line of data is printed
out.

We'll see how powerful these options are, especially when combined with '-e' and '-i'.

9. '-i' is the sysadmin's best friend. It causes all files opened via the "<>"
construct to be edited in-place. When used with an argument as in '-i.bak',
it is the paranoid sysadmin's best friend, because it makes backups of the file being
edited (taking the original name and appending a ".bak" to it to make the name of the
backup). Note that this is especially useful with the '-n' and '-p' arguments, both of
which implicitly use the "<>" construct.

10. '-M' followed by a module name will cause the perl module to be "use"d. The Shell.pm
allows shell commands to be used as perl functions, for example.

$ cd /
$ perl -MShell -e 'print "\nThe contents of the current directory ", pwd(), "\n", ls(-la),"\n";'

The contents of the current directory /

total 903
drwxr-xr-x 17 root root 1024 Nov 24 16:16 .
drwxr-xr-x 17 root root 1024 Nov 24 16:16 ..
drwxr-xr-x 2 root root 1024 Jul 13 1998 .automount
drwxr-xr-x 2 root root 2048 Dec 18 23:50 bin
drwxr-xr-x 2 root root 1024 Jan 10 01:39 boot
lrwxrwxrwx 1 root root 11 Sep 17 20:53 cdrom -> /mnt/cdrom/
-rw------- 1 root root 126976 Sep 20 17:34 core
drwxr-xr-x 2 root root 21504 Jan 16 00:15 dev
drwxr-xr-x 28 root root 3072 Jan 16 00:14 etc
drwxr-xr-x 14 root root 1024 Nov 8 17:10 home
drwxr-xr-x 4 root root 2048 Oct 26 01:03 lib
drwxr-xr-x 2 root root 12288 Sep 1 15:34 lost+found
drwxr-xr-x 8 root root 1024 Sep 16 19:46 mnt
dr-xr-xr-x 5 root root 0 Jan 15 18:41 proc
drwxr-xr-x 11 root root 1024 Jan 16 00:01 root
drwxr-xr-x 3 root root 2048 Nov 10 00:00 sbin
drwxrwxrwt 7 root root 1024 Jan 16 14:48 tmp
drwxr-xr-x 23 root root 1024 Jan 8 16:11 usr
drwxr-xr-x 22 root root 1024 Sep 3 00:51 var
-rw-r--r-- 1 root root 369087 Dec 30 01:53 vmlinuz
-rw-r--r-- 1 root root 368229 Dec 30 00:40 vmlinuz.old


11. '-0' (zero) followed by a up-to-three-digit hex value, causes the input record seperator
to become the character represented by the hex value. We aren't going to get into this too much
here, except for the special case of '-0777'. 777 represents to character, so it causes Perl to
read its input (from Standard Input, usually) in whole. Another special case is '-00', which
causes Perl to read data in one paragraph at a time.

12. '-d' runs your script under the perl debugger. See
http://www.ssc.com/lj/issue49/2484.html for simply the _best_ perl debugger tutorial I have
ever written, er, read.


13. '-v' prints the version of perl you are using. USE VERSION 5.004_04, or higher,
fer crying out loud!


---------------------
- Text manipulation -
---------------------


14. Perl excels at text mangling. To change a the string 'source.syr.edu/~jdimpson'
into 'home.stny.lrun.com/phred' in a group of files, do this:

perl -pi.bak -e 's/source\.syr\.edu\/~jdimpson/home.stny.lrun.com\/phred/g;' *.html

or

perl -pi.bak -e 's#source\.syr\.edu/~jdimpson#home.stny.lrun.com/phred#g;' *.html


15. Another, more complicated example would be to remove all multiple occurences of the
word "the", "a", "an", and "and".

$ cat foo
a a
an an
an a an
the them the
the the the

$ perl -pi.bak -e 'BEGIN{ $word = "the|a|an|and"; }; s/\b($word)(\s+\1)+\b/$1/g ;'

$ cat foo
a
an
an a an
the them the
the


16. Remove comments from C code, foo.c

$ cat foo.c
/* some c code that does nothing */

int main() { /* this is main */

/* nothing here... gaurenteed 100% bug free and Y2K-compliant! */

} /* end of main */

/* This software is copyright 1999 by Jeremy D. Impson. You must pay me $1,000,000.54 drachma
in order to use it. */

$ perl -0777 -pe 's{/\*.*?\*/}{}gs' foo.c


int main() {



}


17. Print first 80 columns of a file.

$ cat foo
0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899


$ perl -pi -e 'chomp(); $_ = substr($_, 0, 80); $_ .= "\n";' foo
$ cat foo
01234567891011121314151617181920212223242526272829303132333435363738394041424344


18. Get rid of beginning and ending spaces

$ perl -pi -e 's/^\s+//; s/\s+$//;'



----------------------
- Command automation -
----------------------

19: Mass renaming of files (change *.htm to *.html):

$ ls *.htm | perl -ne 'chomp(); $file = $_; s/\.htm$/\.html/; print "mv $file $_\n";' | sh

20. Mass renaming of files (uppercase to lowercase):

$ ls
Storable-0.6@3..tar.gz freeamp-1.1.0 onsale.com-account.html
accubyte.com-EDO-order.html ipac-1.00.tar.gz palm
adams.house.txt lug proj
bin onsale.com-EDO-order.html public_html


$ ls *[A-Z]* | perl -ne 'chomp(); $nfile = lc($_); print "mv $_ $nfile\n"' | sh
$ ls
storable-0.6@3..tar.gz freeamp-1.1.0 onsale.com-account.html
accubyte.com-edo-order.html ipac-1.00.tar.gz palm
adams.house.txt lug proj
bin onsale.com-edo-order.html public_html


-----------------
- Miscellaneous -
-----------------

Nice for printing out path variable:

$ perl -e '@path = split /:/, $ENV{'PATH'}; map { print $_,"\n"; } @path;'
/bin
/usr/bin
/usr/local/bin
/usr/bin/X11
/usr/X11R6/bin
/usr/local/gnu/bin
/home/jdimpson/bin
/usr/X11/bin
/usr/andrew/bin
/usr/games
/usr/local/bin/X11



--------------
- References -
--------------

http://language.perl.com

The Perl FAQ (available at above URL) section 3 and 4

man perlrun

END PRIMER