[UFO Chicago] silly bash/awk problem -- fatal: cannot open file

[Neil R. Ormos]

Brian Sobolak wrote:

> I'm trying to run an awk script on a large CSV file.  The script works on
> a column of data, so I want to execute it 78 times, one for each column.

> If I run from the command line, all is well.  If I string them all into a
> script to run one after the other, it fails with the error:

> awk: colWalk.awk:6: fatal: cannot open file
> `/home/bs100/src/awk/fileWalk/Result
> ' for reading (No such file or directory)

> The actual command looks like this:

> awk -f colWalk.awk -v colNum=1 \
> ~/src/awk/fileWalk/Results_June21_Instr_ids_DB2AREAD.csv

> (It all fits on one line obviously.)

> I had changed the file permissions and Results... file is readable by all
> and my script is set to execute.  I don't think that's it.  I am guessing
> that bash is not handling the '-v colNum=1' part correctly which is
> causing it to misinterpret the file name awk needs as an input file.

> Shortening the name made no difference.

> Any ideas?

1.  I was unable to reproduce the problem
    (substituting my local test files for yours,
    mutatis mutandis), using gawk 3.1.6 and GNU
    bash, version 4.1.5(1)-release.

2.  Please send the results of "awk --version" and
    "bash --version".

3.  The error message does not contain the entire
    name of the input file and appears to have a
    newline in it (unless you split the
    error message on two lines to fit in the
    e-mail).  This seems strange.  Could you try a
    different shell to prove (or rule out) your
    hypothesis that bash is mishandling the
    arguments?

4.  Do you have a #! line at the beginning of the
    bash script?  If so, what does it look like?
    If not, how do you know your script is being
    interpreted by bash?  Have you explicitly
    invoked bash?

5.  You might try moving the command-line variable
    assignment in front of the -f option.

6.  You might try quoting the variable assignment.

7.  You might try quoting the name of the input
    file.

8.  You might try using shell redirection via the
    "<" operator instead of naming the input file
    as an argument to awk, unless your program
    requires naming the input file (e.g., your
    need to process multiple files (or multiple
    instances of the same file) in one invocation
    of your program).

9.  You might try replacing the "~" in the
    filename with "$HOME".  While the results are
    usually similar, shells may use subtly
    different mechanisms to handle those
    expansions or substitutions.

10. Are you doing this in cygwyn or the native
    windows environment?

--Neil