How to read multiple files in a loop and check for different conditions
up vote
-1
down vote
favorite
I have lot of files that I will like to run some controls in. In the below example I use 3 files named filexa,filexd,filead.
The file filefo contends in each the format for 3 files mentioned previously.
How I can create a loop to avoid repeating the awk code many times?
I do
Process for filexa file
1) select line 1 in file filefo, which is the same lenght of each line of file filexa
2) run the fist awk to check if each line lenght are the same with filexa file
3) run the second awk to check if column 3 is different than 1
Process for filexd file
1) select line 2 in file filefo, which is the same lenght of each line of file filexd
2) run the fist awk to check if each line lenght are the same with filexd file
3) run the second awk to check if column 4 is equal to 1
Process for filead file
1) select line 3 in file filefo, which is the same lenght of each line of file filead
2) run the fist awk to check if each line lenght are the same with filead file
3) run the second awk to check if column 1 is different than 1000
My problem is because the variables to check are not the same for each file, so it is not easy to do a loop.
Can somebody help me avoid repeating the awk codes and keep a single one using a loop or something similar?
Below the files and process
filexa
1000 2450 1
1000 2500 1
1000 3000 1
filexd
1000 2450 1 3
1000 2500 1 4
1000 3000 1 5
filead
1000 2500
1000 2500
1000 3000
filefo
1000 2450 1
1000 2450 1 3
1000 2500
Process for filexa
awk 'NR==1' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexa
awk '{if($3!~1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexa
Process for filexd
awk 'NR==2' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexd
awk '{if($4==1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexd
Process for filead
awk 'NR==3' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filead
awk '{if($1!~1000){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filead
Thanks in advance
bash awk
asked Nov 10 at 21:52
OXXO
32029
|
show 2 more comments
up vote
-1
down vote
favorite
I have lot of files that I will like to run some controls in. In the below example I use 3 files named filexa,filexd,filead.
The file filefo contends in each the format for 3 files mentioned previously.
How I can create a loop to avoid repeating the awk code many times?
I do
Process for filexa file
1) select line 1 in file filefo, which is the same lenght of each line of file filexa
2) run the fist awk to check if each line lenght are the same with filexa file
3) run the second awk to check if column 3 is different than 1
Process for filexd file
1) select line 2 in file filefo, which is the same lenght of each line of file filexd
2) run the fist awk to check if each line lenght are the same with filexd file
3) run the second awk to check if column 4 is equal to 1
Process for filead file
1) select line 3 in file filefo, which is the same lenght of each line of file filead
2) run the fist awk to check if each line lenght are the same with filead file
3) run the second awk to check if column 1 is different than 1000
My problem is because the variables to check are not the same for each file, so it is not easy to do a loop.
Can somebody help me avoid repeating the awk codes and keep a single one using a loop or something similar?
Below the files and process
filexa
1000 2450 1
1000 2500 1
1000 3000 1
filexd
1000 2450 1 3
1000 2500 1 4
1000 3000 1 5
filead
1000 2500
1000 2500
1000 3000
filefo
1000 2450 1
1000 2450 1 3
1000 2500
Process for filexa
awk 'NR==1' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexa
awk '{if($3!~1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexa
Process for filexd
awk 'NR==2' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexd
awk '{if($4==1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexd
Process for filead
awk 'NR==3' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filead
awk '{if($1!~1000){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filead
Thanks in advance
bash awk
asked Nov 10 at 21:52
OXXO
32029
It isn't clear what you're asking. You talk aboutfilexa,filexd,fileadandfilefo; you then go into examples usingFILE1,FILE2,FILE3. At no point do you explain what the 'format' information infilefomeans. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 offilefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.
– Jonathan Leffler
Nov 11 at 2:07
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Ugh! I guess you could have a singlebashscript that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk ofawkinto a script and doawk -f thatScripton the file, but it's still pretty ugly.
– Mark Setchell
Nov 11 at 7:30
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Gents, please can you help
– OXXO
Nov 11 at 20:34
|
show 2 more comments
up vote
-1
down vote
favorite
up vote
-1
down vote
favorite
I have lot of files that I will like to run some controls in. In the below example I use 3 files named filexa,filexd,filead.
The file filefo contends in each the format for 3 files mentioned previously.
How I can create a loop to avoid repeating the awk code many times?
I do
Process for filexa file
1) select line 1 in file filefo, which is the same lenght of each line of file filexa
2) run the fist awk to check if each line lenght are the same with filexa file
3) run the second awk to check if column 3 is different than 1
Process for filexd file
1) select line 2 in file filefo, which is the same lenght of each line of file filexd
2) run the fist awk to check if each line lenght are the same with filexd file
3) run the second awk to check if column 4 is equal to 1
Process for filead file
1) select line 3 in file filefo, which is the same lenght of each line of file filead
2) run the fist awk to check if each line lenght are the same with filead file
3) run the second awk to check if column 1 is different than 1000
My problem is because the variables to check are not the same for each file, so it is not easy to do a loop.
Can somebody help me avoid repeating the awk codes and keep a single one using a loop or something similar?
Below the files and process
filexa
1000 2450 1
1000 2500 1
1000 3000 1
filexd
1000 2450 1 3
1000 2500 1 4
1000 3000 1 5
filead
1000 2500
1000 2500
1000 3000
filefo
1000 2450 1
1000 2450 1 3
1000 2500
Process for filexa
awk 'NR==1' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexa
awk '{if($3!~1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexa
Process for filexd
awk 'NR==2' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexd
awk '{if($4==1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexd
Process for filead
awk 'NR==3' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filead
awk '{if($1!~1000){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filead
Thanks in advance
bash awk
asked Nov 10 at 21:52
OXXO
32029
I have lot of files that I will like to run some controls in. In the below example I use 3 files named filexa,filexd,filead.
The file filefo contends in each the format for 3 files mentioned previously.
How I can create a loop to avoid repeating the awk code many times?
I do
Process for filexa file
1) select line 1 in file filefo, which is the same lenght of each line of file filexa
2) run the fist awk to check if each line lenght are the same with filexa file
3) run the second awk to check if column 3 is different than 1
Process for filexd file
1) select line 2 in file filefo, which is the same lenght of each line of file filexd
2) run the fist awk to check if each line lenght are the same with filexd file
3) run the second awk to check if column 4 is equal to 1
Process for filead file
1) select line 3 in file filefo, which is the same lenght of each line of file filead
2) run the fist awk to check if each line lenght are the same with filead file
3) run the second awk to check if column 1 is different than 1000
My problem is because the variables to check are not the same for each file, so it is not easy to do a loop.
Can somebody help me avoid repeating the awk codes and keep a single one using a loop or something similar?
Below the files and process
filexa
1000 2450 1
1000 2500 1
1000 3000 1
filexd
1000 2450 1 3
1000 2500 1 4
1000 3000 1 5
filead
1000 2500
1000 2500
1000 3000
filefo
1000 2450 1
1000 2450 1 3
1000 2500
Process for filexa
awk 'NR==1' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexa
awk '{if($3!~1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexa
Process for filexd
awk 'NR==2' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filexd
awk '{if($4==1){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filexd
Process for filead
awk 'NR==3' filefo > tmp-f
awk '
NR == FNR { nf = NF; for (i=1; i<=NF; i++) len[i] = length($i); next }
NF != nf { err = FNR; exit }
{ for (i=1; i<=NF; i++) if (len[i] != length($i)) { err = FNR; exit } }
END {
if (err) {print "ERROR: ", err; exit 1}
}' tmp-f filead
awk '{if($1!~1000){ err = FNR; exit } }
END {
if (err) {print "ERROR,err; exit 1}
} ' filead
Thanks in advance
bash awk
bash awk
asked Nov 10 at 21:52
OXXO
32029
asked Nov 10 at 21:52
OXXO
32029
edited Nov 11 at 6:08
asked Nov 10 at 21:52
OXXO
32029
asked Nov 10 at 21:52
OXXO
32029
asked Nov 10 at 21:52
OXXO
32029
32029
It isn't clear what you're asking. You talk aboutfilexa,filexd,fileadandfilefo; you then go into examples usingFILE1,FILE2,FILE3. At no point do you explain what the 'format' information infilefomeans. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 offilefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.
– Jonathan Leffler
Nov 11 at 2:07
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Ugh! I guess you could have a singlebashscript that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk ofawkinto a script and doawk -f thatScripton the file, but it's still pretty ugly.
– Mark Setchell
Nov 11 at 7:30
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Gents, please can you help
– OXXO
Nov 11 at 20:34
|
show 2 more comments
It isn't clear what you're asking. You talk aboutfilexa,filexd,fileadandfilefo; you then go into examples usingFILE1,FILE2,FILE3. At no point do you explain what the 'format' information infilefomeans. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 offilefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.
– Jonathan Leffler
Nov 11 at 2:07
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Ugh! I guess you could have a singlebashscript that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk ofawkinto a script and doawk -f thatScripton the file, but it's still pretty ugly.
– Mark Setchell
Nov 11 at 7:30
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Gents, please can you help
– OXXO
Nov 11 at 20:34
It isn't clear what you're asking. You talk about
filexa, filexd, filead and filefo; you then go into examples using FILE1, FILE2, FILE3. At no point do you explain what the 'format' information in filefo means. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 of filefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.– Jonathan Leffler
Nov 11 at 2:07
It isn't clear what you're asking. You talk about
filexa, filexd, filead and filefo; you then go into examples using FILE1, FILE2, FILE3. At no point do you explain what the 'format' information in filefo means. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 of filefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.– Jonathan Leffler
Nov 11 at 2:07
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Ugh! I guess you could have a single
bash script that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk of awk into a script and do awk -f thatScript on the file, but it's still pretty ugly.– Mark Setchell
Nov 11 at 7:30
Ugh! I guess you could have a single
bash script that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk of awk into a script and do awk -f thatScript on the file, but it's still pretty ugly.– Mark Setchell
Nov 11 at 7:30
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Gents, please can you help
– OXXO
Nov 11 at 20:34
Gents, please can you help
– OXXO
Nov 11 at 20:34
|
show 2 more comments
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53243774%2fhow-to-read-multiple-files-in-a-loop-and-check-for-different-conditions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It isn't clear what you're asking. You talk about
filexa,filexd,fileadandfilefo; you then go into examples usingFILE1,FILE2,FILE3. At no point do you explain what the 'format' information infilefomeans. You've not explained your processing in terms of what the columns mean. You say you need to read lines 2-4 offilefo, but only show 3 lines in the example data. All-in-all, it is hard for someone to follow what you're trying to do.– Jonathan Leffler
Nov 11 at 2:07
Hi Jonathan, I have updated the question, yes there was a error in the selection of lines for filefo, The awk codes works perfectly. my problem is to do all the 3 processes in single loop to avoid repeated many times same awks process due to small change in awk where i select the line from filefo and second awk where apply a condition. Thanks for your help.
– OXXO
Nov 11 at 5:22
Ugh! I guess you could have a single
bashscript that takes a parameter telling it which of the 3 types of processing is required and the filename. It could then write the appropriate chunk ofawkinto a script and doawk -f thatScripton the file, but it's still pretty ugly.– Mark Setchell
Nov 11 at 7:30
Hi Mark, please can u show me a little how to do it.. Tks
– OXXO
Nov 11 at 7:48
Gents, please can you help
– OXXO
Nov 11 at 20:34