The First time when you login to the remotehost from a localhost, it will display the host key not found message and you can give “yes” to continue. The host key of the remote host will be added under .ssh2/hostkeys directory of your home directory, as shown below.

localhost$ ssh -l jit remotehost.example.com

jit@remotehost.example.com password:

remotehost.example.com$

The Second time when you login to the remote host from the localhost, it will prompt only for the password as the remote host key is already added to the known hosts list of the ssh client.

localhost$ ssh -l jit remotehost.example.com
jit@remotehost.example.com password:
remotehost.example.com$

For some reason, if the host key of the remote host is changed after you logged in for the first time, you may get a warning message as shown below. This could be because of various reasons such as 1) Sysadmin upgraded/reinstalled the SSH server on the remote host 2) someone is doing malicious activity etc., The best possible action to take before saying “yes” to the message below, is to call your sysadmin and identify why you got the host key changed message and verify whether it is the correct host key or not.

localhost$ ssh -l jit remotehost.example.com

jit @remotehost.example.com's password:
remotehost$

4. Debug SSH Client:

Sometimes it is necessary to view debug messages to troubleshoot any SSH connection issues. For this purpose, pass -v (lowercase v) option to the ssh as shown below.

Example without debug message:

        localhost$ ssh -l jit remotehost.example.com
        warning: Connecting to remotehost.example.com failed: No address associated to the name
        localhost$

Example with debug message:

        locaclhost$ ssh -v -l jit remotehost.example.com
        debug: SshConfig/sshconfig.c:2838/ssh2_parse_config_ext: Metaconfig parsing stopped at line 3.
        debug: SshConfig/sshconfig.c:637/ssh_config_set_param_verbose: Setting variable 'VerboseMode' to 'FALSE'.
        debug: SshConfig/sshconfig.c:3130/ssh_config_read_file_ext: Read 17 params from config file.
        debug: Ssh2/ssh2.c:1707/main: User config file not found, using defaults. (Looked for '/home/jit/.ssh2/ssh2_config')
        debug: Connecting to remotehost.example.com, port 22... (SOCKS not used)
        warning: Connecting to remotehost.example.com failed: No address associated to

5. Escape Character: (Toggle SSH session, SSH session statistics etc.)

Escape character ~ get’s SSH clients attention and the character following the ~ determines the escape command.
Toggle SSH Session: When you’ve logged on to the remotehost using ssh from the localhost, you may want to come back to the localhost to perform some activity and go back to remote host again. In this case, you don’t need to disconnect the ssh session to the remote host. Instead follow the steps below.

i. Login to remotehost from localhost: localhost$ssh -l jit remotehost
ii. Now you are connected to the remotehost: remotehost$
iii. To come back to the localhost temporarily, type the escape character ~ and Control-Z. When you type ~ you will not see that immediately on the screen until you press and press enter. So, on the remotehost in a new line enter the following key strokes for the below to work: ~

    remotehost$ ~^Z
    [1]+  Stopped                 ssh -l jit remotehost
    localhost$

iv. Now you are back to the localhost and the ssh remotehost client session runs as a typical unix background job, which you can check as shown below:

    localhost$ jobs
    [1]+  Stopped                 ssh -l jit remotehost

v. You can go back to the remote host ssh without entering the password again by bringing the background ssh remotehost session job to foreground on the localhost

    localhost$ fg %1
    ssh -l jit remotehost
    remotehost$

Part I – Basic Find Commands for Finding Files with Names
1. Find Files Using Name in Current Directory

Find all the files whose name is gene.txt in a current working directory.

# find . -name gene.txt

./gene.txt

2. Find Files Under Home Directory

Find all the files under /home directory with name gene.txt.

# find /home -name gene.txt

/home/gene.txt

3. Find Files Using Name and Ignoring Case

Find all the files whose name is gene.txt and contains both capital and small letters in /home directory.

# find /home -iname gene.txt

./gene.txt
./Gene.txt

4. Find Directories Using Name

Find all directories whose name is Gene in / directory.

# find / -type d -name Gene

/Gene

5. Find fasta Files Using Name

Find all php files whose name is gene.fasta in a current working directory.

# find . -type f -name gene.fasta

./gene.fasta

6. Find all PHP Files in Directory

Find all fasta files in a directory.

# find . -type f -name "*.fasta"

./gene.fasta
./cancer.fasta
./allgene.fasta

Part II – Find Files Based on their Permissions
7. Find Files With 777 Permissions

Find all the files whose permissions are 777.

# find . -type f -perm 0777 -print

8. Find Files Without 777 Permissions

Find all the files without permission 777.

# find / -type f ! -perm 777

9. Find SGID Files with 644 Permissions

Find all the SGID bit files whose permissions set to 644.

# find / -perm 2644

10. Find Sticky Bit Files with 551 Permissions

Find all the Sticky Bit set files whose permission are 551.

# find / -perm 1551

11. Find SUID Files

Find all SUID set files.

# find / -perm /u=s

12. Find SGID Files

Find all SGID set files.

# find / -perm /g+s

13. Find Read Only Files

Find all Read Only files.

# find / -perm /u=r

14. Find Executable Files

Find all Executable files.

# find / -perm /a=x

15. Find Files with 777 Permissions and Chmod to 644

Find all 777 permission files and use chmod command to set permissions to 644.

# find / -type f -perm 0777 -print -exec chmod 644 {} \;

16. Find Directories with 777 Permissions and Chmod to 755

Find all 777 permission directories and use chmod command to set permissions to 755.

# find / -type d -perm 777 -print -exec chmod 755 {} \;

17. Find and remove single File

To find a single file called gene.txt and remove it.

# find . -type f -name "gene.txt" -exec rm -f {} \;

18. Find and remove Multiple File

To find and remove multiple files such as .fa or .gb, then use.

# find . -type f -name "*.fa" -exec rm -f {} \;

OR

# find . -type f -name "*.gb" -exec rm -f {} \;

19. Find all Empty Files

To file all empty files under certain path.

# find /tmp -type f -empty

20. Find all Empty Directories

To file all empty directories under certain path.

# find /tmp -type d -empty

21. File all Hidden Files

To find all hidden files, use below command.

# find /tmp -type f -name ".*"

Part III – Search Files Based On Owners and Groups
22. Find Single File Based on User

To find all or single file called gene.txt under / root directory of owner root.

# find / -user root -name gene.txt

23. Find all Files Based on User

To find all files that belongs to user Rahul under /home directory.

# find /home -user rahul

24. Find all Files Based on Group

To find all files that belongs to group Developer under /home directory.

# find /home -group developer

25. Find Particular Files of User

To find all .txt files of user Rahul under /home directory.

# find /home -user rahul -iname "*.txt"

Part IV – Find Files and Directories Based on Date and Time
26. Find Last 50 Days Modified Files

To find all the files which are modified 50 days back.

# find / -mtime 50

27. Find Last 50 Days Accessed Files

To find all the files which are accessed 50 days back.

# find / -atime 50

28. Find Last 50-100 Days Modified Files

To find all the files which are modified more than 50 days back and less than 100 days.

# find / -mtime +50 –mtime -100

29. Find Changed Files in Last 1 Hour

To find all the files which are changed in last 1 hour.

# find / -cmin -60

30. Find Modified Files in Last 1 Hour

To find all the files which are modified in last 1 hour.

# find / -mmin -60

31. Find Accessed Files in Last 1 Hour

To find all the files which are accessed in last 1 hour.

# find / -amin -60

Part V – Find Files and Directories Based on Size
32. Find 50MB Files

To find all 50MB files, use.

# find / -size 50M

33. Find Size between 50MB – 100MB

To find all the files which are greater than 50MB and less than 100MB.

# find / -size +50M -size -100M

34. Find and Delete 100MB Files

To find all 100MB files and delete them using one single command.

# find / -size +100M -exec rm -rf {} \;

35. Find Specific Files and Delete

Find all .gb files with more than 10MB and delete them using one single command.

# find / -type f -name *.gb -size +10M -exec rm {} \;

Shell history search command

Type history at a shell prompt:
$ history

It will display the list of all used commandline history with an serial number.

To search particular command, enter:
$ history | grep command-name
$ history | egrep -i 'scp|ssh|ftp'
Emacs Line-Edit Mode Command History Searching

To get previous command containing string, hit [CTRL]+[r] followed by search string:

(reverse-i-search):

To get previous command, hit [CTRL]+[p]. You can also use up arrow key.

CTRL-p

To get next command, hit [CTRL]+[n]. You can also use down arrow key.

CTRL-n

fc command

Apart from hostory command there are fc command to extract the command from history. The fc stands for either "find command" or "fix command.

For example list last 10 command, enter:
$ fc -l 10
To list commands 130 through 150, enter:
$ fc -l 130 150
To list all commands since the last command beginning with ssh, enter:
$ fc -l ssh
You can edit commands 1 through 5 using vi text editor, enter:
$ fc -e vi 1 5

Delete command history

The -c option causes the history list to be cleared by deleting all of the entries:
$ history -c

• File System
  ls — list items in current directory
  ls -l — list items in current directory and show in long format to see perimissions, size, and modification date
  ls -a — list all items in current directory, including hidden files
  ls -F — list all items in current directory and show directories with a slash and executables with a star
  ls dir — list all items in directory dir
  cd dir — change directory to dir
  cd .. — go up one directory
  cd / — go to the root directory
  cd ~ — go to to your home directory
  cd - — go to the last directory you were just in
  pwd — show present working directory
  mkdir dir — make directory dir
  rm file — remove file
  rm -r dir — remove directory dir recursively
  cp file1 file2 — copy file1 to file2
  cp -r dir1 dir2 — copy directory dir1 to dir2 recursively
  mv file1 file2 — move (rename) file1 to file2
  ln -s file link — create symbolic link to file
  touch file — create or update file
  cat file — output the contents of file
  less file — view file with page navigation
  head file — output the first 10 lines of file
  tail file — output the last 10 lines of file
  tail -f file — output the contents of file as it grows, starting with the last 10 lines
  vim file — edit file
  alias name 'command' — create an alias for a command
  
• System
  shutdown — shut down machine
  reboot — restart machine
  date — show the current date and time
  whoami — who you are logged in as
  finger user — display information about user
  man command — show the manual for command
  df — show disk usage
  du — show directory space usage
  free — show memory and swap usage
  whereis app — show possible locations of app
  which app — show which app will be run by default
  
• Process Management
  ps — display your currently active processes
  top — display all running processes
  kill pid — kill process id pid
  kill -9 pid — force kill process id pid
  
• Permissions
  ls -l — list items in current directory and show permissions
  chmod ugo file — change permissions of file to ugo - u is the user's permissions, g is the group's permissions, and o is everyone else's permissions. The values of u, g, and o can be any number between 0 and 7.
  7 — full permissions
  6 — read and write only
  5 — read and execute only
  4 — read only
  3 — write and execute only
  2 — write only
  1 — execute only
  0 — no permissions
  chmod 600 file — you can read and write - good for files
  chmod 700 file — you can read, write, and execute - good for scripts
  chmod 644 file — you can read and write, and everyone else can only read - good for web pages
  chmod 755 file — you can read, write, and execute, and everyone else can read and execute - good for programs that you want to share
  
• Networking
  wget file — download a file
  curl file — download a file
  scp user@host:file dir — secure copy a file from remote server to the dir directory on your machine
  scp file user@host:dir — secure copy a file from your machine to the dir directory on a remote server
  scp -r user@host:dir dir — secure copy the directory dir from remote server to the directory dir on your machine
  ssh user@host — connect to host as user
  ssh -p port user@host — connect to host on port as user
  ssh-copy-id user@host — add your key to host for user to enable a keyed or passwordless login
  ping host — ping host and output results
  whois domain — get information for domain
  dig domain — get DNS information for domain
  dig -x host — reverse lookup host
  lsof -i tcp:1337 — list all processes running on port 1337
  
• Searching
  grep pattern files — search for pattern in files
  grep -r pattern dir — search recursively for pattern in dir
  grep -rn pattern dir — search recursively for pattern in dir and show the line number found
  grep -r pattern dir --include='*.ext — search recursively for pattern in dir and only search in files with .ext extension
  command | grep pattern — search for pattern in the output of command
  find file — find all instances of file in real system
  locate file — find all instances of file using indexed database built from the updatedb command. Much faster than find
  sed -i 's/day/night/g' file — find all occurrences of day in a file and replace them with night - s means substitude and g means global - sed also supports regular expressions
  
• Compression
  tar cf file.tar files — create a tar named file.tar containing files
  tar xf file.tar — extract the files from file.tar
  tar czf file.tar.gz files — create a tar with Gzip compression
  tar xzf file.tar.gz — extract a tar using Gzip
  gzip file — compresses file and renames it to file.gz
  gzip -d file.gz — decompresses file.gz back to file
  
• Shortcuts
  ctrl+a — move cursor to beginning of line
  ctrl+f — move cursor to end of line
  alt+f — move cursor forward 1 word
  alt+b — move cursor backward 1 word
  

Address of the bookmark: https://github.com/ventoy/Ventoy

Job description
We are looking for a motivated, self-driven post-doctoral researcher to work with large-scale sequence data analysis. The position is for 24 months and located at Mathematical Statistics, Department of Mathematical Sciences in Erik Kristiansson’s research group. We are focused on methods development for and analysis of next generation DNA sequencing, in particular comparative metagenomics and gene expression analysis (RNA-seq). We have strong interdisciplinary profile and are actively collaborating with several experimental groups, especially within the environmental sciences, ecology, infectious diseases and cancer genomics. More information is available at http://bioinformatics.math.chalmers.se.

The Post-doctoral position is an appointment that offers an opportunity to qualify for further research positions within academia or industry. The majority of your working time is devoted to your own research, normally as a member of a research group. Included in your work is also to take part in supervision of Ph.D. students and M.Sc thesis students. Teaching of undergraduate students may also be included to a small extent.

The employment is limited to a maximum of 2 years (1+1).

Qualifications
The applicant should have Ph.D. degree preferably in bioinformatics, mathematics, statistics, computer science or equivalent by the start of the appointment. Experience from analysis of large-scale data, in particular from next generation DNA sequencing, is highly valued. The applicant should also be proficient in programming (e.g. Python/Java/C) and comfortable with Unix/Linux systems. Interaction with experimental biologists is central and good collaborative skills are therefore important. Fluency in written and spoken English is a strong requirement. As a post-doctoral researcher you are expected to work independently and to be able to supervise/co-supervise PhD and Master’s students.

Application procedure
The application should be marked with Ref 20130126 and written in English. The application should be sent electronically via Chalmers webpage.

Application deadline: September 8, 2013.

For questions, please contact:
Ass prof. Erik Kristiansson, Matematiska Vetenskaper, erik.kristiansson@chalmers.se, +46 31-772 3521, +46 70-5259751.

Chalmers continuously strive to be an attractive employer. Equality and diversity are substantial foundations in all activities at Chalmers.

More at http://pythonhosted.org/pybedtools/

A powerful toolset for genome arithmetic.http://code.google.com/p/bedtools/

The next-generation sequencing included whole genome sequencing(WGS), transcriptome sequencing (whole cDNA sequencing, RNA-seq), digital gene expression sequencing (Tag-Seq), ChIP-Seq, and so on. And there are many sequencing platform to generate sequece, as well know Sanger/ABi(the frist generation), Solexa/illumina, SOLiD/ABi, 454/Roche. But thier sequence format is different, also they have different error type. High quality data is very important for further analysis or data mining. There are many pipeline for raw sequence quality analysis and control with few of process for reporting reads quality statistical details, trimming, filtering, and error correction. Please bookmarks them for the benefits of bioinformatics community.

https://code.google.com/p/biowiki/

https://code.google.com/p/ngs-pipeline/source/browse/#svn%2Ftrunk

NGSand Perl scripts https://code.google.com/hosting/search?q=NGS+perl&projectsearch=Search+projects

NGS and Python scripts https://code.google.com/hosting/search?q=NGS+Python&projectsearch=Search+projects

Address of the bookmark: https://code.google.com/hosting/search?q=bioinformatics&sa=Search

Address of the bookmark: https://github.com/mattharrison/Tiny-Python-3.6-Notebook/blob/master/python.rst

In this tutorial series article, I will explore features and packages of python which are widely used in the big data, NGS, and bioinformatics. I will also walk through a real biological example which shows NGS data processing with the help of python packages and programming.

Python has a couple of points to recommend it to biologists and scientists specifically:

• It's widely used in the scientific community
• It has a couple of very well designed libraries for doing complex scientific computing (although we won't encounter them in this book)
• It lend itself well to being integrated with other, existing tools
• It has features which make it easy to manipulate strings of characters (for example, strings of DNA bases and protein amino acid residues, which we as biologists are particularly fond of)

In general, following are some of the important features of python which makes it a perfect fit for rapid application development.

• Python is interpreted language so the program does not need to be compiled. Interpreter parses the program code and generates the output.
• Python is dynamically typed, so the variables types are defined automatically.
• Python is strongly typed. So the developers need to cast the type manually.
• Less code and more use makes it more acceptable.
• Python is portable, extendable and scalable.

There are two major Python versions, Python 2 and Python 3. Python 2 and 3 are quite different. This tutorial uses Python 3, because it more semantically correct and supports newer features.

I will post tutorial on daily basis on this page. Check the sub-pages on right side.