Using ls command (ls) to sort file names by number.

If you work with a lot of files, you often save the files with numbers for each system.

For example, 

1sample.out

2sample.out

3sample.out

4sample.out

......

......

......

......

10sample.out

11sample.out

12sample.out

.......

.......

Nsample.out

Suppose, you want to extract some information from these files in order from 1sample in order. If you type 'ls', the file 10sample.out will be listed first. You will get list as follow. So the extracted info will be different from what you expect as follow.

10sample.out

11sample.out

12sample.out

...... 

......

15sample.out

1sample.out

2sample.out

.....

.....

Nsample.out

To order file names based on the numbers in the file, use 'ls -v"

Now, the files will be ordered properly.

So, to list the files based on the numbers, use:

ls -v *.out    [to list files with extension .out]

This is a series of post on the "Efficient use of Linux" as well as "Linux Tips for beginners" series. To see all the post, click the link on the right side. 

Using ls command to list all the files in the current directory recursively:

Use:

ls -R

This will list all the files int he current directory recursively.

Check the size of the directory and file (and see the largest in terms of memory)

The command du can be used to check the space (disk usage)

To see the disk space

du -sh * 

To see the disk space and sort it

du -sh * | sort -nr 

To see the disk space and see the top 10 files 

du -sh * | sort -nr | head -n10

To see the disk space and see the top 10 files 

du -sh * | sort -nr | tail -n10

Explanation:

du -s *: Summarizes disk usage of all files

sort -nr: To sort numerically, in reverse order

head -n10: Display first 10 results

tail -n10: Displays last 10 results

h : For human-readable output

To list the top 10 largest files from the current directory, use the following command:

du . | sort -nr | head -n10

du -h . | sort -nr | head -n10 (human readable output)

du, sort -nr and head -n10 serves the same purpose.


To check the diskspace with the control of number of recursive directories.

du -h --max-depth=0 | sort -hr   (only current directory)

du -h --max-depth=1 | sort -hr   (Current directory and one directory depth)


Check free memory in the cluster/supercomputer/linux system

Try:
free -g    (to display the free disk size in GB)

You will get like this.

              total        used        free      shared  buff/cache   available
Mem:              7           6           0           0           1           0

Swap:             7           2           5

Options:
 -b, --bytes       show output in bytes
 -k, --kilo          show output in kilobytes
 -m, --mega      show output in megabytes
 -g, --giga          show output in gigabytes
       --tera          show output in terabytes
 -h, --human    show human-readable output
       --si              use powers of 1000 not 1024
 -l, --lohi           show detailed low and high memory statistics
 -t, --total         show total for RAM + swap
 -s N, --seconds N   repeat printing every N seconds
 -c N, --count N     repeat printing N times, then exit
 -w, --wide          wide output

 --help     display this help and exit
 -V or --version  output version information and exit

This is a series of post on "High Performance Computing (HPC): Everything you need to know before working in a Linux Cluster Environment. To see more post on this series, click here: Important Things to Know to Work in Linux Cluster

Extract only the number from a file name

Some times you may want to extract only the number from a file name, especially when you are working with a huge number of files to do some calculation or to run jobs.

This can be done by (for foo_bar_xyz_file_01_input.in)

echo $f|cut -d_ -f5

Here, $f is the file name.
-d is the delimeter
-f5 is the field where the number is present.

sed tutorial

The stream line editor 'sed' and the 'AWK' are the simple and highly useful languages for processing text files.

Here are some of the tips and tricks to use "sed". Later, I will post on AWK usage.

sed -n '/matchingWord/p' foo.txt

[Note that this is similar to: grep "matchingWord" foo.txt which seems to be very simple for this task. But for many instances, 'sed' may do more.]

sed -i -e 's/match/replacingText/g' hello.txt

Here, the line containing 'match' will be replaced with 'replacingText.




-e : just displays the outcome but don't change the file

-i : change the file (writes in the file)

s  : substitute

g  : is for global substitution. If need only first match, remove g.

Last one is the file name.

You can use this to a bunch of files. 

sed -i -e 's/match/replacingText/g' file*.txt

To delete lines with match pattern

sed -i '/pattern/d' FileName

To replace ":" with " " 

The sed expression s/:/ /g replaces every ":" with a "space".
That is sed substitutes an empty space for every ":" character globally in the file


Sometimes, there may be a need to remove (which is better than replacing a line which contain special characters)

To remove use following onliner

sed '/pattern to match/d' ./filename

To modify the file itself

sed -i '/pattern to match/d' ./filename

Replace the second line (or nth line, first line, third line, etc) using sed:

sed -i '2s/.*/0 5/' c*.in

For other posts like this, see
AWK


For more see following links:
https://stackoverflow.com/a/5410784/6628192

Linux Onliners

To copy multiple files to multiple-different names
for f in *.txt ; do cp -p "$f" " ${f/text/texttobereplaced"; done

To copy a single file to multiple files 
for file in foo*.txt; do cp singlefile "$file" ; done

Note: Here, you have to create the files using touch.


To check the total size of a list of files in a directory
du -ch *.out | tail -n 1


To find all the instances of files with specific extension and delete all of them (recursively).
Before deleting, check all the files that will be deleted
find . -name "*.bak" -type f
Now, to if you are sure that these are the files to delete, run following command.

find . -name "*.bak" -type f -delete

Reference
Ask Ubuntu

Check the number of words/lines/characters in a file

You may know how to check the number of words/characters/lines in a MS Word file. But, how to check these in pdf file or any file?

Here comes the command "wc"

To use this, type the following command (for example to find the number of words in a pdf file).

wc -w fileName.pdf

This will return the number of words in the file.

Linux tips series

To replace the last command with the same arguments

commandName !*  - for all arguments

commandName !$  - for the last argument

commandName !:N - for the Nth argument

Using grep recursively

grep -r "text_to_grep" .

grep -r --include "*.txt" "text_to_grep" .
grep --include="*.csv" -nRHI "text_to_grep" *


After compiling a code, you may search for the executable (if you have a number of files in that directory). In this case, you can use ls -ltr to list all the files as per the time they are created. For example, see blwo.

gfortran -o code.out code.f90

If you run above line you will get the executable code.out in the same directory. But to look at this file, you can type,

ls -ltr

This will list all the files with code.out at the end of the list. This is a quick check for checking whether the executable is created or not. This will be useful when you teach or create a tutorial on coding.


Use top command to check which jobs are running.
Suppose you submit a job to compute in Linux. If you are not sure whether the job is completed or not, use 'top' command to check.

Type on the terminal
top

Now, you will get list of jobs running on the computer.
Then press 1 to see the latest jobs on top.

An easy way to find Latex code

There exists an excellent site on which you can find out the key for most of the $LaTeX$ comments.

The site is Detexify Latex handwriting.

The page says:

"Anyone who works with LaTeX knows how time-consuming it can be to find a symbol in symbols-a4.pdf that you just can't memorize. Detexify is an attempt to simplify this search."

What you have to do is just draw the symbol (on the portion given in the site) for which you want to know the comment.

You will be given the latex-code for that symbol. 

The page says "Philipp Kühl had the initial idea and Daniel Kirsch made it happen". Thanks to them.

sudo apt auto-remove

What is the use of following command?

sudo apt auto-remove

This automatically removes unwanted packages from the linux distro.