xargs command in unix or Linux is a powerful command used in conjunction with find and grep command in UNIX to divide a big list of arguments into small list received from standard input. find and grep command produce long list of file names and we often want to either remove them or do some operation on them but many unix operating system doesn't accept such a long list of argument. UNIX xargs command divide that list into sub-list with acceptable length and made it work. This Unix tutorial is in continuation of my earlier post on Unix like 10 examples of chmod command in Unix and How to update soft link in Linux. If you haven’t read those unit tutorial than check them out. By the way In this tutorial we will see different example of unix xargs command to learn how to use xargs command with find and grep and other unix command and make most of it. Though what you can do with xargs in unix can also be done by using options provided in find but believe xargs is much easy and powerful.
Unix Xargs command example
Following is list of examples of xargs command which shows how useful knowledge of xargs can be. Feel free to copy and use this command and let me know if it didn’t work in any specific Unix operating system like AIX or Solaris.
Xargs Example 1- with and without xargs
in this example of xargs command we will see how output changes with use of xargs command in unix or Linux. Here is the output of find command without xargs first and than with xargs, you can clearly see that multiline output is converted into single line:
devuser@system:/etc find . -name "*bash*"
./bash.bashrc
./bash.bash_logout
./defaults/etc/bash.bashrc
./defaults/etc/bash.bash_logout
./defaults/etc/skel/.bashrc
./defaults/etc/skel/.bash_profile
./postinstall/bash.sh.done
./setup/bash.lst.gz
./skel/.bashrc
./skel/.bash_profile
devuser@system:/etc find . -name "*bash*" | xargs
./bash.bashrc ./bash.bash_logout ./defaults/etc/bash.bashrc ./defaults/etc/bash.bash_logout ./defaults/etc/skel/.bashrc ./defaults/etc/skel/.bash_profile ./postinstall/bash.sh.done ./setup/bash.lst.gz ./skel/.bashrc ./skel/.bash_profile
Xargs Example 2 – xargs and grep
Another common use fo unix xargs command is to first find the files and then look for specific keyword on that file using grep command. here is an example of xargs command that does it
find . -name "*.java" | xargs grep "Stock"
This will first find all java files from current directory or below and than on each java file look for word "Stocks"if you have your development environment setup in Linux or unix this is a great tool to find function references or class references on java files.
Xargs Example 3 – delete temporary file using find and xargs
Another common example of xargs command in unix is removing temporary files from system.
find /tmp -name "*.tmp" | xargs rm
This will remove all .tmp file from /tmp or below directory. xargs in unix is very fast as compared to deleting single file at a time which can also be done by using find command alone. By the way this is also a very popular Unix interview question.
Xargs Example 4 – xargs -0 to handle space in file name
xargs command examples in Unix and Linux - TutorialAbove example of xargs command in unix will not work as expected if any of file name contains space or new line on it. to avoid this problem we use find -print0 to produce null separated file name and xargs-0 to handle null separated items. Here is an example of xargs command in unix which can handle file name with spaces and newline:
find /tmp -name "*.tmp" -print0 | xargs -0 rm
Xargs Example 5 – xargs and cut command in Unix
Though most of xargs examples in unix will be along with find and grep command but xargs is not just limited to this two it can also be used with any command which generated long list of input for example we can use xargs with cut command in unix. In below example of unix xargs we will xargs example with cut command. for using cut command let's first create a .csv file with some data e.g.
devuser@system:/etc cat smartphones.csv
Iphone,Iphone4S
Samsung,Galaxy
LG,Optimus
HTC,3D
Now we will display name of mobile companies from first column using xargs command in one line:
devuser@system:/etc cut -d, -f1 smartphones.csv | sort | xargs
HTC Iphone LG Samsung
xargs Example 6 – command convert muti line output into single line
One more common example of xargs commands in Linux is by converting output of one command into one line. For example you can run any command and then combine xargs to convert output into single line. here is an example xargs in unix which does that.
devuser@system:~/perl ls -1 *.txt
derivatives.txt
futures.txt
fx.txt
options.txt
stock.txt
swaps.txt
devuser@system:~/perl ls -1 *.txt | xargs
derivatives.txt futures.txt fx.txt options.txt stock.txt swaps.txt
Xargs Example 7 - Counting number of lines in each file using xargs and find.
In this example of xargs command in unix we will combine "wc" with xargs and find to count number of lines in each file, just like we did in our previous example with grep where we tried to find specific word in each Java file.
devuser@system:~/perl ls -1 *.txt | xargs wc -l
0 derivatives.txt
2 futures.txt
0 fx.txt
1 options.txt
3 stock.txt
0 swaps.txt
Xargs command Example 7 - Passing subset of arguments to xargs in Linux.
Some commands in unix can only work at certain number of argument e.g. diff command needs two argument. when used with xargs you can use flag "-n" to instruct xargs on how many argument it should pass to given command. this xargs command line option is extremely useful on certain situation like repeatedly doing diff etc. xargs in unix or Linux will continue to pass argument in specified number until it exhaust all input. here is an example of unix xargs command with limited argument:
devuser@system:~/perl ls -1 *.txt | xargs -n 2 echo
derivatives.txt futures.txt
fx.txt options.txt
stock.txt swaps.txt
In this example, xargs is passing just two files at a time to echo as specified with "-n 2" xargs command line option.
Xargs example 9 - avoid "Argument list too long"
xargs in unix or Linux was initially use to avoid "Argument list too long" errors and by using xargs you send sub-list to any command which is shorter than "ARG_MAX" and that's how xargs avoid "Argument list too long" error. You can see current value of "ARG_MAX" by using getconf ARG_MAX. Normally xargs own limit is much smaller than what modern system can allow, default is 4096. You can override xargs sub list limit by using "-s" command line option.
Xargs Example 10 – find –exec vs find + xargs
xargs with find command is much faster than using -exec on find. since -exec runs for each file while xargs operates on sub-list level. to give an example if you need to change permission of 10000 files xargs with find will be almost 10K time faster than find with -exec because xargs change permission of all file at once. For more examples of find and xargs see my post 10 frequently used find command examples in Linux
Important points on xargs command in Unix and Linux
Now let’s revise some important points about xargs command in Unix which is worth remembering :
1. An important point to note about xargs is that it doesn't handle files which has newlines or white space in its name and to avoid this problem one should always use "xargs -0". xargs -o change separator to null character so its important that input feed to xargs is also use null as separator. for example gnu find command use -print0 to produce null separated file names.
2. Xargs command receives command from standard input which is by default separated with space or newlines and
execute those commands, you can still use double quotes or single quote to group commands.
3. If you don't give any command to xargs in unix, default command executed by xargs is /bin/echo and it will simply display file names.
4. One rare problem with xargs is end of file string, by default end of file string is "_" and if this string occurs in input the rest of input is ignored by xargs. Though you can change end of file string by using option "-eof".
5. Use man xargs or xargs --help to get help on xargs online while working in unix or Linux.
In short xargs command in Unix or Linux is an essential tool which enhances functionality of front line commands like find, grep or cut and gives more power to your shell script. These xargs command examples are good start for anyone wants to learn more about xargs command.Though these xargs examples are tested in Linux environment they will applicable for other Unix systems likes Solaris or AIX also. let us know if you face any issue while using these examples.
Unix Xargs command example
Following is list of examples of xargs command which shows how useful knowledge of xargs can be. Feel free to copy and use this command and let me know if it didn’t work in any specific Unix operating system like AIX or Solaris.
Xargs Example 1- with and without xargs
in this example of xargs command we will see how output changes with use of xargs command in unix or Linux. Here is the output of find command without xargs first and than with xargs, you can clearly see that multiline output is converted into single line:
devuser@system:/etc find . -name "*bash*"
./bash.bashrc
./bash.bash_logout
./defaults/etc/bash.bashrc
./defaults/etc/bash.bash_logout
./defaults/etc/skel/.bashrc
./defaults/etc/skel/.bash_profile
./postinstall/bash.sh.done
./setup/bash.lst.gz
./skel/.bashrc
./skel/.bash_profile
devuser@system:/etc find . -name "*bash*" | xargs
./bash.bashrc ./bash.bash_logout ./defaults/etc/bash.bashrc ./defaults/etc/bash.bash_logout ./defaults/etc/skel/.bashrc ./defaults/etc/skel/.bash_profile ./postinstall/bash.sh.done ./setup/bash.lst.gz ./skel/.bashrc ./skel/.bash_profile
Xargs Example 2 – xargs and grep
Another common use fo unix xargs command is to first find the files and then look for specific keyword on that file using grep command. here is an example of xargs command that does it
find . -name "*.java" | xargs grep "Stock"
This will first find all java files from current directory or below and than on each java file look for word "Stocks"if you have your development environment setup in Linux or unix this is a great tool to find function references or class references on java files.
Xargs Example 3 – delete temporary file using find and xargs
Another common example of xargs command in unix is removing temporary files from system.
find /tmp -name "*.tmp" | xargs rm
This will remove all .tmp file from /tmp or below directory. xargs in unix is very fast as compared to deleting single file at a time which can also be done by using find command alone. By the way this is also a very popular Unix interview question.
Xargs Example 4 – xargs -0 to handle space in file name
xargs command examples in Unix and Linux - TutorialAbove example of xargs command in unix will not work as expected if any of file name contains space or new line on it. to avoid this problem we use find -print0 to produce null separated file name and xargs-0 to handle null separated items. Here is an example of xargs command in unix which can handle file name with spaces and newline:
find /tmp -name "*.tmp" -print0 | xargs -0 rm
Xargs Example 5 – xargs and cut command in Unix
Though most of xargs examples in unix will be along with find and grep command but xargs is not just limited to this two it can also be used with any command which generated long list of input for example we can use xargs with cut command in unix. In below example of unix xargs we will xargs example with cut command. for using cut command let's first create a .csv file with some data e.g.
devuser@system:/etc cat smartphones.csv
Iphone,Iphone4S
Samsung,Galaxy
LG,Optimus
HTC,3D
Now we will display name of mobile companies from first column using xargs command in one line:
devuser@system:/etc cut -d, -f1 smartphones.csv | sort | xargs
HTC Iphone LG Samsung
xargs Example 6 – command convert muti line output into single line
One more common example of xargs commands in Linux is by converting output of one command into one line. For example you can run any command and then combine xargs to convert output into single line. here is an example xargs in unix which does that.
devuser@system:~/perl ls -1 *.txt
derivatives.txt
futures.txt
fx.txt
options.txt
stock.txt
swaps.txt
devuser@system:~/perl ls -1 *.txt | xargs
derivatives.txt futures.txt fx.txt options.txt stock.txt swaps.txt
Xargs Example 7 - Counting number of lines in each file using xargs and find.
In this example of xargs command in unix we will combine "wc" with xargs and find to count number of lines in each file, just like we did in our previous example with grep where we tried to find specific word in each Java file.
devuser@system:~/perl ls -1 *.txt | xargs wc -l
0 derivatives.txt
2 futures.txt
0 fx.txt
1 options.txt
3 stock.txt
0 swaps.txt
Xargs command Example 7 - Passing subset of arguments to xargs in Linux.
Some commands in unix can only work at certain number of argument e.g. diff command needs two argument. when used with xargs you can use flag "-n" to instruct xargs on how many argument it should pass to given command. this xargs command line option is extremely useful on certain situation like repeatedly doing diff etc. xargs in unix or Linux will continue to pass argument in specified number until it exhaust all input. here is an example of unix xargs command with limited argument:
devuser@system:~/perl ls -1 *.txt | xargs -n 2 echo
derivatives.txt futures.txt
fx.txt options.txt
stock.txt swaps.txt
In this example, xargs is passing just two files at a time to echo as specified with "-n 2" xargs command line option.
Xargs example 9 - avoid "Argument list too long"
xargs in unix or Linux was initially use to avoid "Argument list too long" errors and by using xargs you send sub-list to any command which is shorter than "ARG_MAX" and that's how xargs avoid "Argument list too long" error. You can see current value of "ARG_MAX" by using getconf ARG_MAX. Normally xargs own limit is much smaller than what modern system can allow, default is 4096. You can override xargs sub list limit by using "-s" command line option.
Xargs Example 10 – find –exec vs find + xargs
xargs with find command is much faster than using -exec on find. since -exec runs for each file while xargs operates on sub-list level. to give an example if you need to change permission of 10000 files xargs with find will be almost 10K time faster than find with -exec because xargs change permission of all file at once. For more examples of find and xargs see my post 10 frequently used find command examples in Linux
Important points on xargs command in Unix and Linux
Now let’s revise some important points about xargs command in Unix which is worth remembering :
1. An important point to note about xargs is that it doesn't handle files which has newlines or white space in its name and to avoid this problem one should always use "xargs -0". xargs -o change separator to null character so its important that input feed to xargs is also use null as separator. for example gnu find command use -print0 to produce null separated file names.
2. Xargs command receives command from standard input which is by default separated with space or newlines and
execute those commands, you can still use double quotes or single quote to group commands.
3. If you don't give any command to xargs in unix, default command executed by xargs is /bin/echo and it will simply display file names.
4. One rare problem with xargs is end of file string, by default end of file string is "_" and if this string occurs in input the rest of input is ignored by xargs. Though you can change end of file string by using option "-eof".
5. Use man xargs or xargs --help to get help on xargs online while working in unix or Linux.
In short xargs command in Unix or Linux is an essential tool which enhances functionality of front line commands like find, grep or cut and gives more power to your shell script. These xargs command examples are good start for anyone wants to learn more about xargs command.Though these xargs examples are tested in Linux environment they will applicable for other Unix systems likes Solaris or AIX also. let us know if you face any issue while using these examples.
No comments:
Post a Comment