Useful Python modules for Physics/Chemists

Numpy

pymatgen : Pymatgen (Python Materials Genomics) is a Python library for materials analysis. The features of this module are: Classes for the representation of Element, Site, Molecule, Structure objects, extensive input/output support, including support for VASP (http://cms.mpi.univie.ac.at/vasp/), ABINIT (http://www.abinit.org/), CIF, Gaussian, XYZ, and many other file formats. Also powerful analysis tools, including generation of phase diagrams, Pourbaix diagrams, diffusion analyses, reactions, etc.
Electronic structure analyses, such as density of states and band structure.

Scipy

Sympy

Pythran
Pythoran is a module to covert python code to C++ code. Highly useful for computational chemistry and physics calculations.





Related links
To know whether a particular module is installed or not, follow this procedure.

User previleges (CHMOD) command

User privileges is one of the important feature. You can change the user privileges of files and directories as follows.

Here CHMOD stands for "change mode". Here is the full wiki page.

To change from normal file to executable file, use:

chmod u+x fileName

To change back to normal file, use:

chmod a-x  fileName

To change the permission 

chmod -R <permission> <dirname>

Once, I wanted to delete the directory recursively (!!!Warning: be careful here, once you delete, you can not recover it). To do this,

chmod -R 777 <dirname>      (then rmdir <dirname>)

Another useful onliner is the following.
Consider a situation where you mistakenly converted some files to executable files.
To covert an executable file to normal file, use:

chmod -x <filename>

To convert multiple files, use:

chmod -x  <foo*.txt>

Here is the list of posts that may be useful if you work in HPC, quantum chemistry, computational materials science, etc.

Working with different shells in Linux

Most of the time we use the default bash shell in the Terminal.

To find the list of shells in your system type

cat /etc/shells

In addition to bash shell, probably widely used shell is c shell.

Following may be the most used open source shells.

• bash
• tcsh/csh
• ksh
• zsh
• fish

Zsh have the features other famous Unix/GNU Linux shells (i.e., bash, tcsh and ksh shell features).

Okay. Now, let us come to csh.

How to install c shell?

In Ubuntu, you can use

sudo apt-get install csh

Now, cshell is installed in your machine.

Now how to use this? Most of the commands you use in bash will work in c shell also. But there are some differences. What are they?

Here is a nice tutorial and list of commands on cshell. Go through int.
Here is the list of main differences between different shells.
Here is another tutorial on cshell.
The Wikibook on C shell scripting can be accessed here.

The 'ls' command in Linux/Unix

The 'ls' command in a Linux system can be used as follows.

To list only the files with no extension
ls !(*.*)              current all the subdirectories
ls -d !(*.*)         only in the current directory
ls -d !(*.tar)         only in the current directory
ls --ignore='*.tar'       without *.tar extension files

ls -I '*.*'

ls | grep -v  "\."      (using grep)

For a comprehensive list of bash commands usage, see here.

Print numbers in bash

To print files with a series of numbers

touch file_{01..10}_input.in
To print a series of files with even numbers

touch file_{02..100..2}_input.in

Using WATCH command to see the updating files

To see the end of the file, we often use tail -n5 or tail (for 10 lines by default). When the file is in progress, we may want to keep on watching the end of the file. This can be done using WATCH command.

The use is as follow:

watch tail -n 12 filename.out
watch -n 2 tail -n 12 filename.out (to see last 12 lines and update every 2 seconds)
watch -n 5 tail -n 12 filename.out (to see last 12 lines and update every 5 seconds)

Notes on Machine Learning

I am writing this notes on the application of machine learning in Sciences.

The applications of machine learning algorithms increases day by day. I wonder what are the specific applications of machine learning in Physics, Chemistry and Biology.

What is machine learning? Is  it possible that a machine can learn? Still we are programming. How it is different from the programming? There is a difference. In programming, you dictate the machine what to do in each step. You are cutting down a big job in to smaller junks of steps containing functions and logical steps, variables, constants, if else statements, for loops, do loops, while conditions etc. You are explicitly programming the computer to do your task. But, in machine learning, you are not programming explicitly. In most of the cases, the algorithms from standard machine learning tools learn from large data and understand about the data. If a new unknown data is given, the machine learning algorithm predict the new data correctly.

First let us see a common example.

If you are using google search engine or amazon online store, you are already benefited by Machine Learning. Because, these companies use Machine Learning algorithms to know user preferences and learned guesses to suggest what the authors want.

Here are some of the courses for learning Machine Learning techniques.

The fields which are useful are.

Even high energy physics.

Some of the chemistry database links are here. These would be useful for machine learning applied to Chemistry and biology problems.

RDKit

ChEMBL

KNIME

PubChem
AFlow library
Materials Project

See here for the full list of database of materials. 

How to 32 bit OS or 64 bit machine

To get the machine type:

uname -m
x86_64 (64 bit)
i686      (32 bit)

You can also use the cammand 'arch'.

For more details see the manual. The manual for uname provides following information.

       -a, --all
              print all information, in the following order, except omit -p and -i if unknown:

       -s, --kernel-name
              print the kernel name

       -n, --nodename
              print the network node hostname

       -r, --kernel-release
              print the kernel release

       -v, --kernel-version
              print the kernel version

       -m, --machine
              print the machine hardware name

       -p, --processor
              print the processor type (non-portable)

       -i, --hardware-platform
              print the hardware platform (non-portable)

       -o, --operating-system
              print the operating system

       --help display this help and exit

       --version
              output version information and exit

Density Functional Theory

In December 1926, Schroedinger published his seminal paper titled "An Undulatory Theory of the Mechanics of Atoms and Molecules" in Physical Review Letters. A few years later, Paul Dirac wrote in his paper (published in Proceedings of the Royal Society) as follow:

"The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble. It therefore becomes desirable that approximate practical methods of applying quantum mechanics should be developed, which can lead to an explanation of the main features of complex atomic systems without too much computation." P. A. M. Dirac (1929)

To get an idea, consider a small molecule $C_{6}H_{6}$)  which contains 6 carbon atoms (6 $\times$ 6 electrons) and 6 hydrogen atoms (6 $\times$ 1 electrons), totaling 36+6=42 electrons.

You may know the Schrodinger's equation for a single particle in a potential $V(x,y,z)$ can be written as

$-\frac{{\hbar}^2}{2m}\dot {{\bigtriangledown}^2}{\psi(x,y,z)}+V(x,y,z)\psi(x,y,z)=E\psi(x,y,z)$

Here, the equation is written in its time-independent form. For time-dependent form see here in the page. Since Kohn-Sham equations were originally developed for ground state systems, now I move forward with time-independent Schrodinger's equation. Later, I will discuss more advanced topics.

(Later P W Anderson wrote a paper opposing Dirac's view. See that paper titled "More is different" published in Science Magazine. 

OK. Let us take the words of Dirac. Approximate practical methods of applying quantum mechanics have to be developed. 

What are the approaches? Firs of all what we should know? Total energy. What is total energy? The total energy (kinetic energy + potential energy+ any other energy) associated with the total system that include nucleus and electrons. 

As a simple approach, consider that there is only kinetic energy and potential energy contribution and set all other contribution to zero. Then, 

Total energy = kinetic energy + potential energy.

Let us assume $C_{6}H_{6}$) molecule. For this molecule, as we saw above, there are 42 electrons. So, considering 3 co-ordinates for each electrons, $42 \times 3$ becomes 126 coordinates (see this description from Prof. Hardy Gross's lecture to understand this).

(for electron) simply kinetic energy = $\frac{p^{2}}{2m}$, where $p$ is the moment of the electron and $m$ is the mass of the electron. What is the potential energy? We can calculate the potential energy of a single electron on positive or negative charges. But, our system is a interacting system. right? Lets come to that interacting system later. Now let us focus on the non-interacting system. Consider that the electrons are non-interacting with each other and they interact only with the nucleus (positive ions). Then, we need to account only the nucleus-electron attraction and the kinetic energy of the electron. This is exactly what Thomas-Fermi model does in electronic structure calculation (look for post on this). This omit the electron-electron interaction completely and considers that the electrons are like gases.

Now, what we need is to calculate the Hamiltonian for a given system (let us construct for C6H6). How to construct the kinetic energy functional (in terms of density) and electron-neucleus potential functional? Let us see further in this.

Kinetic enregy = $\frac{p^{2}}{2m}$.

Now, this is in momentum space. We need to convert this kinetic energy in to a functional of density. Let us do that.

The exchange correlation functional is an important correction to the the total energy which gives about 3% of total energy. Eventhough it is small percentage, it is crucial in dermining (....)

(this page will be updated regularly).

Checking the status of the submitted job in HPC

The submitted jobs in PBS queue system can be checked whether they are in queue still, or running, or completed, etc. The command to use (in PBS software) is qstate.

Job id            Name             User              Time Use S Queue
----------------  ---------------- ----------------  -------- - -----
306.clr     jobname              user              10:36:59 R worke

Commands to use are:
qstat    (lists all the jobs with some details)
qstat -n  (list all the jobs with total number of CPUs running the jobs
qstat -u user (list all the jobs submitted by the user)


To see a list of things to know (for a newbie) to use high performance cluster, click here.