User previleges (CHMOD) command

User privileges is one of the important feature. You can change the user privileges of files and directories as follows.

Here CHMOD stands for "change mode". Here is the full wiki page.

To change from normal file to executable file, use:

chmod u+x fileName

To change back to normal file, use:

chmod a-x  fileName

To change the permission 

chmod -R <permission> <dirname>

Once, I wanted to delete the directory recursively (!!!Warning: be careful here, once you delete, you can not recover it). To do this,

chmod -R 777 <dirname>      (then rmdir <dirname>)

Another useful onliner is the following.
Consider a situation where you mistakenly converted some files to executable files.
To covert an executable file to normal file, use:

chmod -x <filename>

To convert multiple files, use:

chmod -x  <foo*.txt>

Here is the list of posts that may be useful if you work in HPC, quantum chemistry, computational materials science, etc.

Working with different shells in Linux

Most of the time we use the default bash shell in the Terminal.

To find the list of shells in your system type

cat /etc/shells

In addition to bash shell, probably widely used shell is c shell.

Following may be the most used open source shells.

• bash
• tcsh/csh
• ksh
• zsh
• fish

Zsh have the features other famous Unix/GNU Linux shells (i.e., bash, tcsh and ksh shell features).

Okay. Now, let us come to csh.

How to install c shell?

In Ubuntu, you can use

sudo apt-get install csh

Now, cshell is installed in your machine.

Now how to use this? Most of the commands you use in bash will work in c shell also. But there are some differences. What are they?

Here is a nice tutorial and list of commands on cshell. Go through int.
Here is the list of main differences between different shells.
Here is another tutorial on cshell.
The Wikibook on C shell scripting can be accessed here.

The 'ls' command in Linux/Unix

The 'ls' command in a Linux system can be used as follows.

To list only the files with no extension
ls !(*.*)              current all the subdirectories
ls -d !(*.*)         only in the current directory
ls -d !(*.tar)         only in the current directory
ls --ignore='*.tar'       without *.tar extension files

ls -I '*.*'

ls | grep -v  "\."      (using grep)

For a comprehensive list of bash commands usage, see here.

Print numbers in bash

To print files with a series of numbers

touch file_{01..10}_input.in
To print a series of files with even numbers

touch file_{02..100..2}_input.in

Using WATCH command to see the updating files

To see the end of the file, we often use tail -n5 or tail (for 10 lines by default). When the file is in progress, we may want to keep on watching the end of the file. This can be done using WATCH command.

The use is as follow:

watch tail -n 12 filename.out
watch -n 2 tail -n 12 filename.out (to see last 12 lines and update every 2 seconds)
watch -n 5 tail -n 12 filename.out (to see last 12 lines and update every 5 seconds)

Notes on Machine Learning

I am writing this notes on the application of machine learning in Sciences.

The applications of machine learning algorithms increases day by day. I wonder what are the specific applications of machine learning in Physics, Chemistry and Biology.

What is machine learning? Is  it possible that a machine can learn? Still we are programming. How it is different from the programming? There is a difference. In programming, you dictate the machine what to do in each step. You are cutting down a big job in to smaller junks of steps containing functions and logical steps, variables, constants, if else statements, for loops, do loops, while conditions etc. You are explicitly programming the computer to do your task. But, in machine learning, you are not programming explicitly. In most of the cases, the algorithms from standard machine learning tools learn from large data and understand about the data. If a new unknown data is given, the machine learning algorithm predict the new data correctly.

First let us see a common example.

If you are using google search engine or amazon online store, you are already benefited by Machine Learning. Because, these companies use Machine Learning algorithms to know user preferences and learned guesses to suggest what the authors want.

Here are some of the courses for learning Machine Learning techniques.

The fields which are useful are.

Even high energy physics.

Some of the chemistry database links are here. These would be useful for machine learning applied to Chemistry and biology problems.

RDKit

ChEMBL

KNIME

PubChem
AFlow library
Materials Project

See here for the full list of database of materials. 

How to 32 bit OS or 64 bit machine

To get the machine type:

uname -m
x86_64 (64 bit)
i686      (32 bit)

You can also use the cammand 'arch'.

For more details see the manual. The manual for uname provides following information.

       -a, --all
              print all information, in the following order, except omit -p and -i if unknown:

       -s, --kernel-name
              print the kernel name

       -n, --nodename
              print the network node hostname

       -r, --kernel-release
              print the kernel release

       -v, --kernel-version
              print the kernel version

       -m, --machine
              print the machine hardware name

       -p, --processor
              print the processor type (non-portable)

       -i, --hardware-platform
              print the hardware platform (non-portable)

       -o, --operating-system
              print the operating system

       --help display this help and exit

       --version
              output version information and exit

Density Functional Theory

In December 1926, Schroedinger published his seminal paper titled "An Undulatory Theory of the Mechanics of Atoms and Molecules" in Physical Review Letters. A few years later, Paul Dirac wrote in his paper (published in Proceedings of the Royal Society) as follow:

"The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble. It therefore becomes desirable that approximate practical methods of applying quantum mechanics should be developed, which can lead to an explanation of the main features of complex atomic systems without too much computation." P. A. M. Dirac (1929)

To get an idea, consider a small molecule $C_{6}H_{6}$)  which contains 6 carbon atoms (6 $\times$ 6 electrons) and 6 hydrogen atoms (6 $\times$ 1 electrons), totaling 36+6=42 electrons.

You may know the Schrodinger's equation for a single particle in a potential $V(x,y,z)$ can be written as

$-\frac{{\hbar}^2}{2m}\dot {{\bigtriangledown}^2}{\psi(x,y,z)}+V(x,y,z)\psi(x,y,z)=E\psi(x,y,z)$

Here, the equation is written in its time-independent form. For time-dependent form see here in the page. Since Kohn-Sham equations were originally developed for ground state systems, now I move forward with time-independent Schrodinger's equation. Later, I will discuss more advanced topics.

(Later P W Anderson wrote a paper opposing Dirac's view. See that paper titled "More is different" published in Science Magazine. 

OK. Let us take the words of Dirac. Approximate practical methods of applying quantum mechanics have to be developed. 

What are the approaches? Firs of all what we should know? Total energy. What is total energy? The total energy (kinetic energy + potential energy+ any other energy) associated with the total system that include nucleus and electrons. 

As a simple approach, consider that there is only kinetic energy and potential energy contribution and set all other contribution to zero. Then, 

Total energy = kinetic energy + potential energy.

Let us assume $C_{6}H_{6}$) molecule. For this molecule, as we saw above, there are 42 electrons. So, considering 3 co-ordinates for each electrons, $42 \times 3$ becomes 126 coordinates (see this description from Prof. Hardy Gross's lecture to understand this).

(for electron) simply kinetic energy = $\frac{p^{2}}{2m}$, where $p$ is the moment of the electron and $m$ is the mass of the electron. What is the potential energy? We can calculate the potential energy of a single electron on positive or negative charges. But, our system is a interacting system. right? Lets come to that interacting system later. Now let us focus on the non-interacting system. Consider that the electrons are non-interacting with each other and they interact only with the nucleus (positive ions). Then, we need to account only the nucleus-electron attraction and the kinetic energy of the electron. This is exactly what Thomas-Fermi model does in electronic structure calculation (look for post on this). This omit the electron-electron interaction completely and considers that the electrons are like gases.

Now, what we need is to calculate the Hamiltonian for a given system (let us construct for C6H6). How to construct the kinetic energy functional (in terms of density) and electron-neucleus potential functional? Let us see further in this.

Kinetic enregy = $\frac{p^{2}}{2m}$.

Now, this is in momentum space. We need to convert this kinetic energy in to a functional of density. Let us do that.

The exchange correlation functional is an important correction to the the total energy which gives about 3% of total energy. Eventhough it is small percentage, it is crucial in dermining (....)

(this page will be updated regularly).

Checking the status of the submitted job in HPC

The submitted jobs in PBS queue system can be checked whether they are in queue still, or running, or completed, etc. The command to use (in PBS software) is qstate.

Job id            Name             User              Time Use S Queue
----------------  ---------------- ----------------  -------- - -----
306.clr     jobname              user              10:36:59 R worke

Commands to use are:
qstat    (lists all the jobs with some details)
qstat -n  (list all the jobs with total number of CPUs running the jobs
qstat -u user (list all the jobs submitted by the user)


To see a list of things to know (for a newbie) to use high performance cluster, click here. 

Git and GitHub Tutorial

What is a version control and usage?

When we write a project with a number of files, it is not easy to maintain. Some times, we might have edited the source file a lot but later we might realize that we want the earlier version of the source code. Git does that.

Git is a version control system which can be used to manage changes made to the files and directories in any project. There are a number of version control systems out there. But, git is famous may be because of its developer (who?).

Anything saved to git is ever lost and at any time you can go back to earlier versions and also can clearly see which version of the program generated which version of the output.

It is also useful when you work with collaborators. Git will notify the changes as well as confilicts with others. So, overwriting problem will not occur.

"Git can synchronize work done by different people on different machines, so it scales as your team does."

Version control uses: It is not just for developing code. It can be used for writing books, books, journal papers, parameter sets (anything that change over time or need to be shared)

The first step:

Create a directory (My-Repo) in your home directory or anywhere in your linux system.

Now type, git status in that directory.

Now you will get.
fatal: Not a git repository (or any of the parent directories): .git
This shows that, the current directory is not a git repository. To start that directory as a repository, you have to use

git init

Now,

Initialized empty Git repository in /home/user/path/to/directory/My_Repo/.git/

will be displayed. This means, the current directory has been initalized as Git repository.

git status

On branch master

Initial commit

nothing to commit (create/copy files and use "git add" to track)

This means that we are in the master branch of the repository.

Now, create an empty file inside this directory (i.e., repository). You can use touch hello.txt

Now, again type

git status

Now, you will get,

On branch master

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        hello.txt

nothing added to commit but untracked files present (use "git add" to track)

This means that (git says) a new file hello.txt is created in the repository (now the current directory is a repository) and the file is not tracked by the git.

To make the git to track the created file hello.txt, you have to use the command

The git has following status of the files.

untracked: Git is not tracking this file yet, usually indicating that the file is new.
tracked: The file is part of the Git snapshot and Git is tracking changes to it.
unstaged: A tracked file has changes but it has not been staged for commit.
staged: The file is ready to be committed.
deleted: The file has been deleted and should be removed from Git.

Now, the steps to follow is
git add hello.txt

This upgrade the status of the hello.txt to staged.

Now, if you type

git status in the repository,

On branch master

Initial commit

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

        new file:   hello.txt

will be displayed. This means that the file hello.txt is tracked.

We can more files and just use git add (fileName). This will add all the files to staged mode. Or you can use
git add .
This command will all the files in the current directory to track mode.

So far, your file is just staged. To make it track,

git commit -m "This is a new repository"

The message inside the "" is for reference purpose.

Now, you will get following message.

git commit -m "This is to learn git and github"
[master (root-commit) 8de3ed7] This is to learn git and github
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 hello.txt
 create mode 100644 hello2.txt

Now, at this stage, don't worry about connecting to GitHub and configuring your email ID to github etc. We will see them later.


Now, we will see little bit about the git config.

To see the status of your configure,

cat .git/config

This will display the configure name and emailID.

You can also try
git config --global --list

This will display the name and email associated with the repository.

If not name and emailID is not set so far, follow this.

git config --global user.name "yourName"
git config --global user.email "yourid@gmail.com"

Now, if you hit
git config --global --list

you will get

YouName
user.email=yourid@gmail.com
user.name=yourName

The basic workflow is as follow:

git status
git add
git commit
git push

If your configuration correct, your changes would be pushed to the remote repository.
Till git commit, you can work locally and saving the track of each modification by using add and commit. But, if you want to export the changes to the remote repository, you have to use git push.

In my case, first time I am trying to push to the remote repository.

git push

warning: push.default is unset; its implicit value has changed in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the traditional behavior, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

Since Git 2.0, Git defaults to the more conservative 'simple'
behavior, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

Now, I am prompted to enter the username and password:

Username for 'https://github.com': userName
Password for 'https://userName@github.com':
To https://github.com/userName/github-workflow.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://github.com/ssthurai/github-workflow.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

This shows that the push is not success. Let us see more about this push later. Keep in touch.

When we commit a change in a file, we are prompted for inserting comment. For me, the default browser "nano" came. However, I prefer 'vim'. To change to vim as a default editor

git config --global core.editor "vim"