BOL: Related items

Linux Sort Commands for Bioinformatics

Rahul Nayak — Sat, 31 May 2014 15:41:16 -0500

Almost all the scripting languages such as Perl, Python etc have built-in sort, but unfortunately none of them are as flexible as sort command. But one when it come to space efficiency GNU sort stands at the top. It can sort a 20Gb file with less than 2Gb memory. It is not trivial to implement so powerful a sort by yourself.

sort a space-delimited file based on its first column, then the second if the first is the same, and so on:
sort input.txt

sort a huge file (GNU sort ONLY):
sort -S 1500M -t $HOME/tmp input.txt > sorted.txt

sort starting from the third column, skipping the first two columns:
sort +2 input.txt

sort the second column as numbers, descending order; if identical, sort the 3rd as strings, ascending order:
sort -k2,2nr -k3,3 input.txt

sort starting from the 4th character at column 2, as numbers:
sort -k2.4n input.txt

More Linxu sort command information

If you have any sort commands you'd like to share, please add them to our comments section below. For more help, you can also type:

man sort

or

sort --help

on your Unix/Linux system.

Tetra-Nucleotide Analysis

Jit — Thu, 04 May 2017 05:07:41 -0500

A tetra-nucleotide is a fragment of DNA sequence with 4 bases (e.g. AGTC or TTGG). Pride et al. (2003) showed that the frequency of tetra-nucleotides in bacterial genomes contain useful, albeit weak, phylogenetic signals. Even though tetra-nucleotide analysis (TNA) utilizes the information of whole genome, it is evident that it cannot replace other alignment-based phylogenetic methods such as OrthoANI or 16S rRNA phylogeny. However, TNA can be useful for phylogenetic characterization when whole genome or 16S rRNA gene information is not available. For example, a partial genomic fragment obtained from a metagenome can be identified by TNA (Teeling et al., 2004). TNA is also fast enough that it can be used as a search engine against a large genome database.

Address of the bookmark: https://chunlab.wordpress.com/tetra-nucleotide-analysis/

GEnView: A phylogeny based comparative genomics software to analyze the genetic environment of genes

Abhi — Tue, 28 Dec 2021 01:49:03 -0600

A phylogeny based comparative genomics software to analyze the genetic environment of genes. The user can select one or several taxa and provide one or several reference protein(s). Genomes and plasmids (based on user choice) will be downloaded from the NCBI Assembly/NR database and searched for the respective gene. Alternatively, custom genomes can be provided. User selected stretches (20kbp by default) of the genes genetic environment are extracted, annotated and aligned between all genomes. The sequences are then visualized, enabling comparison of synteny and gene content.

More at https://pubmed.ncbi.nlm.nih.gov/34951622/

Address of the bookmark: https://github.com/EbmeyerSt/GEnView

Tree illustrating the lack of interchromosomal rearrangement of the microchromosomes.

BioJoker — Mon, 26 Nov 2018 04:20:33 -0600

Tree illustrating the lack of interchromosomal rearrangement of the microchromosomes. No interchromosomal microchromosome fusions from the avian ancestor unless otherwise stated (macrochromosomal fusions not listed). The overall pattern of microchromosome stability and rearrangement across the species is illustrated

Jarvis et al. (2014)

Reference https://link.springer.com/article/10.1007/s00412-018-0685-6

Senior Bioinformatician (Assembly) Moore Aquatic Symbiosis Project Tree of Life

Sat, 02 Oct 2021 00:28:30 -0500

You will have some previous experience with genome bioinformatics or other large scale scientific data analysis, or a newly qualified graduate student with data science skills interested in DNA sequence data. While desirable, previous experience with DNA sequencing data is not strictly necessary for the position. We have a strong publication record and culture of producing open data resources and open source software development. This role requires an investigative and solution-oriented mindset and excellent communication skills to work effectively within large national and international consortia.

More at https://jobs.sanger.ac.uk/vacancy/senior-bioinformatician-assembly-moore-aquatic-symbiosis-project-tree-of-life-458923.html

TMRCA Calculator

BioStar — Wed, 03 Feb 2021 05:07:30 -0600

This program calculates the probability that two people have a certain number of generations between them, based on the standard infinite alleles formula of Walsh. It calculates both the probability of being at an exact number of generations back to the Most Recent Common Ancestor (MRCA) of a certain pair of people and the cumulative probability that the actual number of generations is less than a certain value. Note that the convention using generations is changed from an earlier version of this calculator which used "transmission events". It can list both result types in a table or graph. In either case the horizontal axis stops at the point where the cumulative probability reaches 95% or 10 generations, whichever is longer, or an absolute max of 50,000. Beyond 90% the calculation becomes inaccurate.

https://clandonaldusa.org/index.php/tmrca-calculator

Address of the bookmark: https://clandonaldusa.org/index.php/tmrca-calculator

MIKE: an ultrafast, assembly-, and alignment-free approach for phylogenetic tree construction

Abhi — Mon, 08 Apr 2024 06:19:52 -0500

MIKE (MinHash-based k-mer algorithm). This algorithm is designed for the swift calculation of the Jaccard coefficient directly from raw sequencing reads and enables the construction of phylogenetic trees based on the resultant Jaccard coefficient. Simulation results highlight the superior speed of MIKE compared to existing state-of-the-art methods. We used MIKE to reconstruct a phylogenetic tree, incorporating 238 yeast, 303 Zea, 141 Ficus, 67 Oryza, and 43 Saccharum spontaneum samples. MIKE demonstrated accurate performance across varying evolutionary scales, reproductive modes, and ploidy levels, proving itself as a powerful tool for phylogenetic tree construction.

Address of the bookmark: https://github.com/Argonum-Clever2/mike

Find certain files/documents in Linux OS

Rahul Nayak — Sun, 06 Apr 2014 23:56:18 -0500

As bioinformatician I know the fact that we usually handle the large dataset and lost in the huge numbers of files and folders. In order to search the missing file a strong search command is required. The Linux Find Command is one of the most important and much used command in Linux sytems. Find command used to search and locate list of files and directories based on conditions you specify for files that match the arguments. Find can be used in variety of conditions like you can find files by permissions, users, groups, file type, date, size and other possible criteria.

Through this article we are sharing our day-to-day Linux find command experience and its usage in the form of examples. In this article we will show you the most used 35 Find Commands examples in Linux. We have divided the section into Five parts from basic to advance usage of find command.

Part I – Basic Find Commands for Finding Files with Names
1. Find Files Using Name in Current Directory

Find all the files whose name is gene.txt in a current working directory.

# find . -name gene.txt

./gene.txt

2. Find Files Under Home Directory

Find all the files under /home directory with name gene.txt.

# find /home -name gene.txt

/home/gene.txt

3. Find Files Using Name and Ignoring Case

Find all the files whose name is gene.txt and contains both capital and small letters in /home directory.

# find /home -iname gene.txt

./gene.txt
./Gene.txt

4. Find Directories Using Name

Find all directories whose name is Gene in / directory.

# find / -type d -name Gene

/Gene

5. Find fasta Files Using Name

Find all php files whose name is gene.fasta in a current working directory.

# find . -type f -name gene.fasta

./gene.fasta

6. Find all PHP Files in Directory

Find all fasta files in a directory.

# find . -type f -name "*.fasta"

./gene.fasta
./cancer.fasta
./allgene.fasta

Part II – Find Files Based on their Permissions
7. Find Files With 777 Permissions

Find all the files whose permissions are 777.

# find . -type f -perm 0777 -print

8. Find Files Without 777 Permissions

Find all the files without permission 777.

# find / -type f ! -perm 777

9. Find SGID Files with 644 Permissions

Find all the SGID bit files whose permissions set to 644.

# find / -perm 2644

10. Find Sticky Bit Files with 551 Permissions

Find all the Sticky Bit set files whose permission are 551.

# find / -perm 1551

11. Find SUID Files

Find all SUID set files.

# find / -perm /u=s

12. Find SGID Files

Find all SGID set files.

# find / -perm /g+s

13. Find Read Only Files

Find all Read Only files.

# find / -perm /u=r

14. Find Executable Files

Find all Executable files.

# find / -perm /a=x

15. Find Files with 777 Permissions and Chmod to 644

Find all 777 permission files and use chmod command to set permissions to 644.

# find / -type f -perm 0777 -print -exec chmod 644 {} \;

16. Find Directories with 777 Permissions and Chmod to 755

Find all 777 permission directories and use chmod command to set permissions to 755.

# find / -type d -perm 777 -print -exec chmod 755 {} \;

17. Find and remove single File

To find a single file called gene.txt and remove it.

# find . -type f -name "gene.txt" -exec rm -f {} \;

18. Find and remove Multiple File

To find and remove multiple files such as .fa or .gb, then use.

# find . -type f -name "*.fa" -exec rm -f {} \;

OR

# find . -type f -name "*.gb" -exec rm -f {} \;

19. Find all Empty Files

To file all empty files under certain path.

# find /tmp -type f -empty

20. Find all Empty Directories

To file all empty directories under certain path.

# find /tmp -type d -empty

21. File all Hidden Files

To find all hidden files, use below command.

# find /tmp -type f -name ".*"

Part III – Search Files Based On Owners and Groups
22. Find Single File Based on User

To find all or single file called gene.txt under / root directory of owner root.

# find / -user root -name gene.txt

23. Find all Files Based on User

To find all files that belongs to user Rahul under /home directory.

# find /home -user rahul

24. Find all Files Based on Group

To find all files that belongs to group Developer under /home directory.

# find /home -group developer

25. Find Particular Files of User

To find all .txt files of user Rahul under /home directory.

# find /home -user rahul -iname "*.txt"

Part IV – Find Files and Directories Based on Date and Time
26. Find Last 50 Days Modified Files

To find all the files which are modified 50 days back.

# find / -mtime 50

27. Find Last 50 Days Accessed Files

To find all the files which are accessed 50 days back.

# find / -atime 50

28. Find Last 50-100 Days Modified Files

To find all the files which are modified more than 50 days back and less than 100 days.

# find / -mtime +50 –mtime -100

29. Find Changed Files in Last 1 Hour

To find all the files which are changed in last 1 hour.

# find / -cmin -60

30. Find Modified Files in Last 1 Hour

To find all the files which are modified in last 1 hour.

# find / -mmin -60

31. Find Accessed Files in Last 1 Hour

To find all the files which are accessed in last 1 hour.

# find / -amin -60

Part V – Find Files and Directories Based on Size
32. Find 50MB Files

To find all 50MB files, use.

# find / -size 50M

33. Find Size between 50MB – 100MB

To find all the files which are greater than 50MB and less than 100MB.

# find / -size +50M -size -100M

34. Find and Delete 100MB Files

To find all 100MB files and delete them using one single command.

# find / -size +100M -exec rm -rf {} \;

35. Find Specific Files and Delete

Find all .gb files with more than 10MB and delete them using one single command.

# find / -type f -name *.gb -size +10M -exec rm {} \;

Phylogenetic for Bioinformatics

Jitendra Narayan — Tue, 16 Jul 2013 03:50:30 -0500

Biologists estimate that there are about 5 to 100 million species of organisms living on Earth today. Evidence from morphological, biochemical, and gene sequence data suggests that all organisms on Earth are genetically related, and the genealogical relationships of living things can be represented by a vast evolutionary tree, the Tree of Life. The Tree of Life then represents the phylogeny of organisms, i. e., the history of organismal lineages as they change through time.
Every living organism contains DNA, RNA, and proteins. Closely related organisms generally have a high degree of agreement in the molecular structure of these substances, while the molecules of organisms distantly related usually show a pattern of dissimilarity. Molecular phylogeny uses such data to build a "relationship tree" that shows the probable evolution of various organisms. Not until recent decades, however, has it been possible to isolate and identify these molecular structures.
phylogenetics is the study of evolutionary relatedness among various groups of organisms (for example, species or populations), which is discovered through molecular sequencing data and morphological data matrices. In other word, Phylogenetics, the science of phylogeny, is one part of the larger field of systematics, which also includes taxonomy. Taxonomy is the science of naming and classifying the diversity of organisms Molecular phylogeny is the use of the structure of molecules to gain information on an organism's evolutionary relationships. The result of a molecular phylogenetic analysis is expressed in a so-called phylogenetic tree.

The evolutionary connections between organisms are represented graphically through phylogenetic trees. Due to the fact that evolution takes place over long periods of time that cannot be observed directly, biologists must reconstruct phylogenies by inferring the evolutionary relationships among present-day organisms.
Application of the techniques that make this possible can be seen in the very limited field of human genetics, such as the ever more popular use of genetic testing to determine a child's paternity, as well as the emergence of a new branch of criminal forensics focused on genetic evidence.
The effect on traditional scientific classification schemes in the biological sciences has been dramatic as well. Work that was once immensely labor- and materials-intensive can now be done quickly and easily, leading to yet another source of information becoming available for systematic and taxonomic appraisal. This particular kind of data has become so popular that taxonomical schemes based solely on molecular data may be encountered. Proponents even claim that taxonomy was previously based on morphology alone, which of course is utter fable.

For additional information on phylogenetics, see list of Phylogenetics Resources on the Internet.

Phylogeny and Reconstructing Phylogenetic Trees: http://aleph0.clarku.edu/~djoyce/java/Phyltree/cover.html
the CBRG and Department of Statistics Phylogeny tutorial: http://www.compbio.ox.ac.uk/tutorials/phylogeny/
TUTORIAL: PHYLOGENETIC ANALYSIS USING PARSIMONY:http://home.cc.umanitoba.ca/~psgendb/GDE/phylogeny/parsimony/phylip.parsimony.html

PHYLIP: http://www.umanitoba.ca/afs/plant_science/psgendb/doc/Phylip/main.html
An Introduction to Molecular Phylogeny: http://bibiserv.techfak.uni-bielefeld.de/gcb04/tutorials/hoef-emden/GCB04Tut.pdf

How to make a phylogenetic tree: http://www.hiv.lanl.gov/content/sequence/TUTORIALS/TREE_TUTORIAL/Treetutorial.html
Phylogenetic Trees: http://cnx.org/content/m11052/latest/
Phylogeny by Ron Shamir: http://www.cs.tau.ac.il/~rshamir/algmb/01/scribe08/lec08.pdf
Introduction to Phylogeny: http://www.utm.edu/departments/cens/biology/rirwin/391/391Phylog.htm
Lecturer notes on Phylogeny: http://www.sbc.su.se/~bens/course_material/phylocourse1/lecture2.pdf
Principles and Practice of Phylogenetic Systematics:http://www.faculty.biol.ttu.edu/Strauss/Phylogenetics/LectureNotes.htm

Inferring phylogenetic trees: http://www.cis.hut.fi/Opinnot/T-61.6070/slides2008/pres_6070.pdf

Lecture Notes

Chapter 1 - The Diversity, Classification, and Evolution of Vertebrates:http://academic.emporia.edu/mooredwi/nathist/chap1.htm

Algorithms for Phylogenetic Reconstructions:http://lectures.molgen.mpg.de/Algorithmische_Bioinformatik_WS0405/phylogeny_script.pdf

Phylogeny.fr is a free, simple to use web service dedicated to reconstructing and analysing phylogenetic relationships between molecular sequences. Phylogeny.fr runs and connects various bioinformatics programs to reconstruct a robust phylogenetic tree from a set of sequences. For more detail : http://www.phylogeny.fr/version2_cgi/index.cgi

A Brief Tutorial on Phylogenetics
http://bioss.ac.uk/~dirk/talks/tutorial_phylogenetics.pdf

A Brief Tutorial on Phylogenetics Human Rabbit Chicken
http://bioss.ac.uk/~dirk/talks/psnup_tutorial_phylogenetics.pdf

Phylogenetic Tree Computation Tutorial Overview
http://pga.lbl.gov/Workshop/April2002/lectures/Olken.pdf

MrBayes: A program for the Bayesian inference of phylogeny
http://golab.unl.edu/teaching/SBseminar/manual.pdf

Web sites providing software for the construction of phylogenetic trees