Bash script filter files

Or is there a different method to filter and display logs based on unique size only. Do you know the file size, or do you just want do find files with identical size and exclude them? Testing, the! Benjamin W, I would like to do the latter, though I can manually ID file sizes, there are 48 thousand logs, so "find files with identical size and exclude" is my goal — vol.

Add a comment. Active Oldest Votes. One solution is: find. The awk part prints the path of the first file with a given size only. Mischa Mischa 2, 20 20 silver badges 15 15 bronze badges. But the OP wants all the files for which no other file with the same size exists.

This appears to be quite close, I added this to the end xargs ls -lSr -h to view it with file sizes, and there are no subdirectories in this location. Active Oldest Votes. To ignore more than one file add an -I before each filename. Improve this answer. Sudheer Sudheer 4, 4 4 gold badges 22 22 silver badges 27 27 bronze badges.

If you use the long-form option --ignore you can extend that to glob patterns e. This is what I needed and worked. Damodar Bashyal Damodar Bashyal 4 4 silver badges 8 8 bronze badges.

The reason it doesn't work unquoted is because of shell expansion, i. What's actually happening is that it ignores the first. Double-quoting worked for me as well. Thanks verboze — user Kartikey Tanna Kartikey Tanna 2 2 silver badges 6 6 bronze badges.

As steeldriver says , from your stated goals it looks like you don't want your script to consider all the files in the current directory, as find. Still, it may well be reasonable to use find for this, and one benefit of doing so is that if you do need to modify it to operate recursively later, that will be very easy. On the other hand, it's often unwise to undertake extra effort to prepare for features you might need to add later, since it's hard to predict what features those will actually be.

In the rest of this answer, I'll explore that possibility. I'll give what might be a full solution, but you may or may not decide it's acceptable for your needs. You'll likely want to modify it in some way. My goal isn't to provide a polished script, but instead to demonstrate techniques you can use.

You'll likely get other answers that demonstrate approaches like shell loops and checking each file with [ -d filename ] as steeldriver suggested. If you want to use find , you can pass on the arguments your script received to find as roots for its search in place of where you currently wrote. This would do the wrong thing if some of the filename arguments to your script start with - , since they could be interpreted specially by find as options, predicates, or actions.

But most commands treat arguments with leading - characters specially; usually it is sufficient to document this behavior for other users and to ensure you don't accidentally pass such paths to your script. If you really want to consider an entry in the current directory named like -foo , you can pass its name as. For simplicity, I will first consider how to solve the easier problem of outputting lines for all arguments passed to the script that are names of directories, no matter how many entries they contain i.

One thing to consider is what happens if the user doesn't pass any arguments. Then find sees no roots to start with. When this happens, it behaves as if you passed. Not all find implementations treat the absence of roots as an implied. You should handle this It's reasonable to use -exec to call another command that will count the files in each directory, and your approach to that is on the right track.

But this should go before the -printf action or whatever other action you use to cause find to output filenames , so that this action only happens for directories that pass the test.

If you choose to count the files by counting how many lines of output an ls command produces, then that ls command should be invoked with the -q option. This is because filenames can contain weird characters you might not expect, including newline characters. You may also want to use the -1 option, because although it is redundant when ls outputs to a pipe as here , it clearly documents your intent that ls emit one line per file.

In general you should avoid parsing ls , but since you don't care about the actual filenames and -q ensures you get one line per file, it is okay. But I'll show a different technique instead. In order to filter text, one has to use a text filtering tool such as awk. You can think of awk as a programming language of its own. But for the scope of this guide to using awk , we shall cover it as a simple command line filtering tool. Where 'script' is a set of commands that are understood by awk and are execute on file, filename.

It works by reading a given line in the file, makes a copy of the line and then executes the script on the line. This is repeated on all the lines in the file. In the following examples, we shall focus on the meta characters that we discussed above under the features of awk.

It will match strings containing localhost , localnet , lines , capable , as in the example below:. The next example matches strings starting with either K or k followed by T :. It allows you to take the character following it as a literal that is to say consider it just as it is.