sed -i 's/[^,]*,//' file.csv

Note the [^,]* bit, which matches everything that is not a comma. Don’t use .*, because this will greedily match commas, too. (Usually, cut is useful for these sort of things, but cut cannot readily replace the file in place.)

I wanted a regular expression to match the pattern of mutt mail edit buffers, to apply mail-mode, but I did not want to match the muttrc and mutt.hooks files I have.

Mail edit buffers get a pattern that starts with “mutt”, followed by a combination of dashes, letters and numbers. Examples:

mutt-lauren-ad34AD-
muttadR12
muttadrsd

The pattern mutt[-0-9a-zA-Z]+$ matches these just fine, but it would also match muttrc. So I want a regex that looks for at least three occurrences from the character class described in the brackets. Generally, this is done using {3,} (using the {m,n} pattern to match at least m and at most n occurrences). (You can match exactly n occurrences, by using {3}).

In Emacs, this didn’t work, and it turns out I had to escape the curly brackets twice: mutt[-0-9a-zA-Z]\\{3,\\}$.

Here’s the full section in my .emacs file:

(defun mutt-edit-hook ()
  (setq fill-column 70)
  (setq make-backup-files nil)
  )

(add-to-list 'auto-mode-alist '("mutt[-0-9a-zA-Z]\\{3,\\}$" . mail-mode))
(add-hook 'mail-mode-hook 'turn-on-auto-fill)
(add-hook 'mail-mode-hook 'mutt-edit-hook)

ls *.!(Rnw|tex)

Run from a shell script, however, this would give the error:

line 5: syntax error near unexpected token `('

It turns out on ubuntu, the shell option extglob is set for an interactive shell, but not for the non-interactive shell that is used when you run a script. You can see whether “extglob” is on like this, either from a commandline or by putting the line into your script:

shopt extglob

If it’s off and you want it on, put into your script:

shopt -s extglob

An alternative (that you might not want, I’m not sure what else it might affect) is to change the opening line of your script to read

#!/bin/bash -i

This runs your script in an interactive shell and should thus have the same behavior as the commandline. Generally, a neat trick is to use the -x option to debug shell scripts:

#!/bin/bash -x

An alternative way to get files whose names match a certain pattern is of course to use find. Here, you can either use limited globbing patterns with the -name option, and combine several patterns using -o, like this:

find . -name "*.tex" -o -name "*.txt"

Find also has a -regex option. If you use that, you probably want to use -regex-type=posix-extended to get a behavior that is more similar to extended regular expressions that you might know, for example from sed. Additionally, you have to watch out that find needs the regular expression to match the entire filename including the directory. Here is an example for this:

audrey:~/tmp/testdir$ ls
blah.tex  blah.txt
audrey:~/tmp/testdir$ find . -regex "blah\.tex"
audrey:~/tmp/testdir$ find . -regex "\./blah\.tex"
./blah.tex

Here is an example for the difference between the default and the extended regex pattern behavior:

audrey:~/tmp/testdir$ find . -regex "\./blah\.(tex|txt)"

finds nothing, but:

audrey:~/tmp/testdir$ find . -regextype posix-extended -regex "\./blah\.(tex|txt)"
./blah.tex
./blah.txt

You could alternatively get the desired behavior with the default regex by escaping the grouping parentheses and the “|” operator:

audrey:~/tmp/testdir$ find . -regex "\./blah\.\(tex\|txt\)"
./blah.tex
./blah.txt

But that starts getting really hard to read.

Delete all leading whitespace from line (tabs and spaces):

sed 's/^[ \t]*//'

Delete whitespace at end of line:

sed 's/[ \t]*$//'

Delete leading and trailing whitespace:

sed 's/^[ \t]*//;s/[ \t]*$//'

So here’s the first example. I have a file containing assignments done by students, 100 of them, all one after another in a single text file. I’ve graded them and want to e-mail them back. I’ll use sed and mutt to do this conveniently from the commandline.

The text file looks like this:

studentname@mail.server.com,Firstname Lastname,
The above line is followed by the student's answer to the assignment,
bla bla bla,
several lines of text. At the end, the grade is out of five points:
> 2/5.
secondstudentname@some.mailserver.com,Firstname2 Lastname2,
The first assignment is followed by the next assignment, which again ends in
a grade, and so on for 100 students.
> 4/5.

First, I want to get rid of the student name and the two commas on the line with the e-mail address, so that the e-mail address is on a line by itself. The sed command:

s|\(@.*\),.*,|\1|

More specifically, this looks for a regexp matching the pattern of an ampersand followed by any sort of stuff ( “.*” ), a comma, again anything, followed by another comma, and replaces it by reinserting the first found pattern that we’ve specified with the two escaped brackets “\(” and “\)”.
Next, I turn each line with an e-mail address into a mutt command to send anything up to the next “EOF” to that e-mail address, with a sensible subject line. The sed command:

s|\(.*@.*\)|mutt \1 -s "psych 153 a3" << EOF|

We insert two newlines and “EOF” after each grade. The grade is identified as a pattern of “/5.”, and the period needs to be escaped (otherwise, it would be a regexp match for any character):

s|/5.|/5.\n\nEOF|

You can put all the above statemens into a file, maybe call it “sedfile”, backup your work and then run it with “in place” replacement, i.e., the original file gets overwritten:

cp ass3 ass3.bkp
sed -i -f sedfile ass3

Finally, make the file executable and run it:

chmod 700 ass3
./ass3