BOL: Related items

Basic command-line to run BLAST

Shruti Paniwala — Wed, 14 Mar 2018 05:10:34 -0500

The goal of this tutorial is to run you through a demonstration of the command line, which you may not have seen or used much before.

All of the commands below can copy/pasted.

Install software

Copy and paste the following commands

sudo apt-get update && sudo apt-get -y install python ncbi-blast+

This updates the software list and installs the Python programming language and NCBI BLAST+.

Get Data

Grab some data to play with. Grab some cow and human RefSeq proteins:

wget ftp://ftp.ncbi.nih.gov/refseq/B_taurus/mRNA_Prot/cow.1.protein.faa.gz
wget ftp://ftp.ncbi.nih.gov/refseq/H_sapiens/mRNA_Prot/human.1.protein.faa.gz

This is only the first part of the human and cow protein files - there are 24 files total for human.

The database files are both gzipped, so lets unzip them

gunzip *gz
ls

Take a look at the head of each file:

head cow.1.protein.faa
head human.1.protein.faa

These are protein sequences in FASTA format. FASTA format is something many of you have probably seen in one form or another – it’s pretty ubiquitous. It’s just a text file, containing records; each record starts with a line beginning with a ‘>’, and then contains one or more lines of sequence text.

Note that the files are in fasta format, even though they end if ”.faa” instead of the usual ”.fasta”. This NCBI’s way of denoting that this is a fasta file with amino acids instead of nucleotides.

How many sequences are in each one?

grep -c '^>' cow.1.protein.faa
grep -c '^>' human.1.protein.faa

This grep command uses the c flag, which reports a count of lines with match to the pattern. In this case, the pattern is a regular expression, meaning match only lines that begin with a >.

This is a bit too big, lets take a smaller set for practice. Lets take the first two sequences of the cow proteins, which we can see are on the first 6 lines

head -6 cow.1.protein.faa > cow.small.faa

BLAST

Now we can blast these two cow sequences against the set of human sequences. First, we need to tell blast about our database. BLAST needs to do some pre-work on the database file prior to searching. This helps to make the software work a lot faster. Because you installed your own version of the sotware, you need to tell the shell where the software is located. Use the full path and the makeblastdb command:

makeblastdb -in human.1.protein.faa -dbtype prot
ls

Note that this makes a lot of extra files, with the same name as the database plus new extensions (.pin, .psq, etc). To make blast work, these files, called index files, must be in the same directory as the fasta file.

blastp [-h] [-help] [-import_search_strategy filename]
[-export_search_strategy filename] [-task task_name] [-db database_name]
[-dbsize num_letters] [-gilist filename] [-seqidlist filename]
[-negative_gilist filename] [-negative_seqidlist filename]
[-entrez_query entrez_query] [-db_soft_mask filtering_algorithm]
[-db_hard_mask filtering_algorithm] [-subject subject_input_file]
[-subject_loc range] [-query input_file] [-out output_file]
[-evalue evalue] [-word_size int_value] [-gapopen open_penalty]
[-gapextend extend_penalty] [-qcov_hsp_perc float_value]
[-max_hsps int_value] [-xdrop_ungap float_value] [-xdrop_gap float_value]
[-xdrop_gap_final float_value] [-searchsp int_value]
[-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking]
[-matrix matrix_name] [-threshold float_value] [-culling_limit int_value]
[-best_hit_overhang float_value] [-best_hit_score_edge float_value]
[-window_size int_value] [-lcase_masking] [-query_loc range]
[-parse_deflines] [-outfmt format] [-show_gis]
[-num_descriptions int_value] [-num_alignments int_value]
[-line_length line_length] [-html] [-max_target_seqs num_sequences]
[-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo]
[-use_sw_tback] [-version]

Now we can run the blast job. We will use blastp, which is appropriate for protein to protein comparisons.

blastp -query cow.small.faa -db human.1.protein.faa

This gives us a lot of information on the terminal screen. But this is difficult to save and use later - Blast also gives the option of saving the text to a file.

    blastp -query cow.small.faa -db human.1.protein.faa -out cow_vs_human_blast_results.txt
ls

Take a look at the results using less. Note that there can be more than one match between the query and the same subject. These are referred to as high-scoring segment pairs (HSPs).

less cow_vs_human_blast_results.txt

So how do you know about all the options, such as the flag to create an output file? Lets also take a look at the help pages. Unfortunately there are no man pages (those are usually reserved for shell commands, but some software authors will provide them as well), but there is a text help output

blastp -help

To scroll through slowly

blastp -help | less

To quit the less screen, press the q key.

Parameters of interest include the -evalue (Default is 10?!?) and the -outfmt

Lets filter for more statistically significant matches with a different output format:

blastp \
-query cow.small.faa \
-db human.1.protein.faa \
-out cow_vs_human_blast_results.tab \
-evalue 1e-5 \
-outfmt 7

I broke the long single command into many lines with by “escaping” the newline. That forward slash tells the command line “Wait, I’m not done yet!”. So it waits for the next line of the command before executing.

Check out the results with less.

Lets try a medium sized data set next

head -199 cow.1.protein.faa > cow.medium.faa

What size is this db?

grep -c '^>' cow.medium.faa

Lets run the blast again, but this time lets return only the best hit for each query.

blastp \
-query cow.medium.faa \
-db human.1.protein.faa \
-out cow_vs_human_blast_results.tab \
-evalue 1e-5 \
-outfmt 6 \
-max_target_seqs 1

Summary

Review:

command line programs such as blast use flags to get information about how and what to do
blast options can be found by typing blastp -help
break a command up over many lines by using `` to “escape” the new line

Blastn

blastn [-h] [-help] [-import_search_strategy filename]
[-export_search_strategy filename] [-task task_name] [-db database_name]
[-dbsize num_letters] [-gilist filename] [-seqidlist filename]
[-negative_gilist filename] [-negative_seqidlist filename]
[-entrez_query entrez_query] [-db_soft_mask filtering_algorithm]
[-db_hard_mask filtering_algorithm] [-subject subject_input_file]
[-subject_loc range] [-query input_file] [-out output_file]
[-evalue evalue] [-word_size int_value] [-gapopen open_penalty]
[-gapextend extend_penalty] [-perc_identity float_value]
[-qcov_hsp_perc float_value] [-max_hsps int_value]
[-xdrop_ungap float_value] [-xdrop_gap float_value]
[-xdrop_gap_final float_value] [-searchsp int_value]
[-sum_stats bool_value] [-penalty penalty] [-reward reward] [-no_greedy]
[-min_raw_gapped_score int_value] [-template_type type]
[-template_length int_value] [-dust DUST_options]
[-filtering_db filtering_database]
[-window_masker_taxid window_masker_taxid]
[-window_masker_db window_masker_db] [-soft_masking soft_masking]
[-ungapped] [-culling_limit int_value] [-best_hit_overhang float_value]
[-best_hit_score_edge float_value] [-window_size int_value]
[-off_diagonal_range int_value] [-use_index boolean] [-index_name string]
[-lcase_masking] [-query_loc range] [-strand strand] [-parse_deflines]
[-outfmt format] [-show_gis] [-num_descriptions int_value]
[-num_alignments int_value] [-line_length line_length] [-html]
[-max_target_seqs num_sequences] [-num_threads int_value] [-remote]
[-version]

DESCRIPTION
Nucleotide-Nucleotide BLAST 2.7.0+

Mike Ritchie Lab

Wed, 02 Oct 2013 15:25:45 -0500

Mike Ritchie Lab primary research focus is the detection of susceptibility genes for common diseases such as cancer, diabetes, hypertension, and cardiovascular disease, among others. The approaches will involve the development and application of new statistical methods with a focus on the detection of gene-gene interactions associated with human disease.

Gene expression and protein expression patterns between normal and non-normal tissues is a growing area of research that may lead to the identification of candidate genes for understanding the etiology of common, complex diseases.

Lab homepage @ http://ritchielab.psu.edu/ritchielab/

New Layout for BLAST ftp Database Site

Jit — Tue, 21 Jan 2020 11:57:11 -0600

As announced previously, the new default database version for BLAST+ is dbV5. To complete this transition, the ftp database site will be updated to support this change. We expect this change to happen around February 4^th, please adjust your scripts or procedures accordingly.

Here is a list of what is changing:

All databases at the root level will be dbV5.
The dbV5 file naming, “_v5” will be removed. Databases with no “_vX” descriptor will be dbV5.
dbV4 tarballs will be renamed with "_v4", files included in tarball will not be renamed.
dbV4 databases will be moved to a v4 subdirectory.
As of 1/13/20 the Cloud directory will be frozen with no more new entries.
The will be no more updates to dbV4 databases.
The FASTA directory will contain nr, nt, swissprot, and pdbaa files.

If you have any questions or concerns, please contact blast-help@ncbi.nlm.nih.gov

Research Associate @ ICGEB, New Delhi.

Wed, 09 Oct 2013 13:49:20 -0500

Applications are invited for Research Associate position in the DBT Sponsored Bioinformatics Infrastructure Facility at ICGEB, New Delhi.

Essential requirements: Experience of using bioinformatics tools.

Experience of working in Linux. Basic knowledge of computer network administration.

Desirable: Knowledge of Linux installation/administration and proficiency in either of the following:

Shell/PERL/Java/Python/VB/Oracle/MySQL/C/CUDA.

Qualification: PhD. or First class M.Sc degree in Bioinformatics or Biotechnology/life science with specialization in Bioinformatics.

Fellowships: Rs 22,000/- with HRA for PhD qualified, Rs 16000/- with HRA for NET/BET/BINC/GATE qualified and 12000/- with HRA for non NET qualified applicants.

Interested candidates may send their complete biodata along with a write-up of their experience and suitability for the position to Dr. Dinesh Gupta by email only to dinesh@icgeb.res.in within 15 days of publication of this advertisement. Kindly mark the email with subject “Application for BIF-RA-2013”

Closing date for applications: 18 October 2013

Only short listed candidates will be invited for an interview at ICGEB.

No TA/DA will be paid for attending the interview.

Advertisement: http://www.icgeb.org/tl_files/Vacancies/BIF-RA-Advt.pdf

SRF/JRF/RA @ UNIVERSITY OF HYDERABAD

Mon, 14 Oct 2013 07:49:11 -0500

SCHOOL OF CHEMISTRY, UNIVERSITY OF HYDERABAD

Applications on plain paper along with details of CV (relevant photocopies of their
qualifications/experience and reprints of published work to be attached) are invited from qualified candidates for Research Fellowship in CSIR- sponsored research project.

JRF/SRF/RA (one vacancy)

CSIR sponsored “In silico design, identification and in vitro validation of lead molecule inhibitors to Bcr-Abl kinase”

JRF: M.Sc in Chemistry/ Bioinformatics/ Biotechnology with I division and NET or GATE qualified

SRF: M.Sc in chemistry/ Bioinformatics/ Biotechnology with at least two years of post- M.Sc research experience as evidenced from published papers in standard refereed journals in relevant area

RA: PhD in chemistry/ Bioinformatics/ Biotechnology with research experience in
relevant area.

As per CSIR guidelines

Notes:
1) You may visit the University of Hyderabad website www.uohyd.ernet.in to learn more about the University of Hyderabad.
2) Applicants should note that the appointment to be made is purely temporary and there is no right for claiming for any regular appointment in the University.
3) No TA/DA will be paid for attending the interview or at the time of joining the post, if selected.
4) The application should be submitted by post/courier/in-person to the address given below on or before November 1st 2013.

Prof. Lalitha Guruprasad
W-103, Gurbakhsh Singh Building
School of Chemistry
University of Hyderabad
Hyderabad- 500 046
5) Short-listed candidates will be called for interview at a short notice.

Advertisement: http://www.uohyd.ac.in/images/recruitment/chemisry_advt_101013.pdf

New BLAST Core Nucleotide Database (core_nt)

LEGE — Tue, 13 Aug 2024 07:12:53 -0500

The Core Nucleotide Database (core_nt) is now the default nucleotide BLAST database. Core_nt is also available on the command line. You get faster searches & more focused results.

Core_nt contains the same eukaryotic transcript and gene-related sequences as nt. The core_nt database is nt without most eukaryotic chromosome sequences. Most nucleotide BLAST searches with core_nt will be similar to the nt database. However, core_nt is better than nt for accomplishing your most common BLAST search goals, such as identifying gene-related sequences like transcript sequences and complete bacterial chromosomes. This is because, in recent years, nt has acquired more low-relevance, non-annotated, and non-gene content.

Learn more: https://ncbiinsights.ncbi.nlm.nih.gov/2024/07/18/new-blast-core-nucleotide-database/

Research Fellow in Bioinformatics @ Queen's University Belfast -Institute for Global Food Security, School of Biological Sciences

Thu, 17 Oct 2013 04:33:02 -0500

Ref: 13/102900

Available immediately until 30th November 2015, to work on the development of bioinformatics approaches to aid analysis of data derived from the metabolomic profiling of biological matrices. The successful applicant will lead research activities on an FP7 funded EU-wide collaborative project aimed at establishing biomarker-based strategies for high throughput diagnostic screening. Key tasks will involve multivariate analysis of large datasets, bioinformatic-based selection and validation of identified markers, construction of metabolomic spectral profile databases and development of machine learning/database searching approaches amenable to analytical screening techniques. This position will offer the opportunity to travel and undertake work with project collaborators based in the Republic of Ireland and Europe.

Informal enquiries may be directed to Dr Terry McGrath, email: terry.mcgrath@qub.ac.uk.

Anticipated interview date: Thursday 31st October 2013
Salary scale: £30,424 – £39,649 per annum (including contribution points)
Closing date: Monday 21st October 2013

Telephone (028) 90973044 FAX: (028) 90971040 or e-mail on personnel@qub.ac.uk

The University is committed to equality of opportunity and to selection on merit. It therefore welcomes applications from all sections of society and particularly welcomes applications from people with a disability.

Fixed term contract posts are available for the stated period in the first instance but in particular circumstances may be renewed or made permanent subject to availability of funding.

More @ https://hrwebapp.qub.ac.uk/tlive_webrecruitment/wrd/run/ETREC107GF.open?VACANCY_ID=5616943npO&WVID=6273090Lgx&LANG=USA

BLAST options, setting and defaults

Jit — Mon, 10 Dec 2018 08:29:37 -0600

BLAST stands for Basic Local Alignment Search Tool and was developed by Altschul et al. (1990) and significantly improved by Altschul et al. (1997). It is a very fast search algorithm that is used to separately search protein or DNA databases. BLAST is best used for sequence similarity searching, rather than for motif searching. For searches using a query sequence of fewer than twenty residues, PatMatch is the best choice. Another sequence alignment tool that may yield different results from BLAST, and may be useful for motif searching, is FASTA. To search nonplant datasets, try NCGR BLAST or NCBI BLAST.

A fairly complete on-line guide to BLAST searching can be found at the NCBI BLAST Help Manual. For a theoretical overview of BLAST, see the NCBI BLAST Course. Additional information can be found in the BLAST 2.0 Release Notes

	BLASTN	BLASTP	BLASTX	TBLASTN	TBLASTX	PSIBLAST
Gap opening penalty: cost to open a gap [integer]	default = 5	default = 11 limited values are supported	default = 11 limited values are supported	default = 11 limited values are supported	default = 11 limited values are supported	default = 5
Gap extension penalty: cost to extend a gap [integer]	default = 2	default = 1 a 0 in this field means to use the default	default = 1 a 0 in this field means to use the default	default = 1 a 0 in this field means to use the default	default = 1 a 0 in this field means to use the default	default = 2
Nucleic match: reward for a match in the BLAST portion of run [integer]	default = 1	n/a	n/a	n/a	n/a	default = 1
Nucleic mismatch: penalty for a mismatch in the blast portion of run [integer]	default = -3	n/a	n/a	n/a	n/a	default = -3
Expectation value: (E) [real]	default = 10.0	default = 10.0	default = 10.0	default = 10.0	default = 10.0	default = 10.0
Word size: the size of the initial word that must be matched between the database and the query sequence	default = 11	default = 3	default = 3	default = 3	default = 3	default = 11
Max scores: Number of one-line descriptions (V) [Integer]	default = 25	default = 25	default = 25	default = 25	default = 25	default = 25
Max alignments: number of alignments to show (B) [integer]	default = 15	default = 15	default = 15	default = 15	default = 15	default = 15
Query filter: filter applied to the query sequence	default = DUST	default = SEG	default = SEG	default = SEG	default = SEG	default = DUST
Query genetic code: genetic code to be used in BLASTX translation of the query	n/a	n/a	default = universal	default = universal	default = universal	n/a
Matrix: substitution matrix to be used for amino acid comparisons	no default	default = blosum62	default = blosum62	default = blosum62	default = blosum62	no default

Supported and Suggested Values for Gap Open and Extension in BLASTP, BLASTX, TBLASTN, and TBLASTX

Gaps Open	Gap Extension
10	1
10	2
11	1
8	2
9	2

Address of the bookmark: https://www.arabidopsis.org/Blast/BLASToptions.jsp

JRF @ NATIONAL JALMA INSTITUTE OF LEPROSY AND OTHER MYCOBACTERIAL DISEASES

Mon, 28 Oct 2013 10:42:48 -0500

NATIONAL JALMA INSTITUTE OF LEPROSY AND OTHER MYCOBACTERIAL DISEASES

(INDIAN COUNCIL OF MEDICAL RESEARCH)

P.O BOX 101,
Dr. M. Miyazaki Marg,
Tajganj, Agra - 282001

Applications are invited for a walk-in interview to be held in the Seminar Hall of the on 15th November, 2013, 9:30 am for temporary positions of JRF, Lab Technician and Field attendant in a ICMR funded project entitled "Elucidating the strain differentiation and transmission dynamics of M. leprae through simple sequence repeats ISSR-PCR marker"

1. JRF (one Post)

Essential qualification: Candidates with M.Sc/IVI.Tech or equivalent degree in any life science related subjects with UGC-CSIR/ICMR/DBT-Net qualified

Desirable qualification: Experience in Molecular Biology/Computational Biology will be preferred.

Age. Maximum 28 years as on 11.11.2013. Age relaxation as per GOI rules.

Emoluments: Rs. 6,000 + 20% HRA per Month

2. Lab Technician (One Post)

Essential Qualification: 12th with DMLT/B.SCA4.SC in Life sciences

Desirable qualification: Experience in Molecular Biology/Computational Biology will be preferred.

Age: Maximum 30 years as on 11.11.2013. Age relaxation as per GOI rules.

Emoluments: Rs13,760/ Per Month

3. Field Attendant (One Post)

Essential Qualification: 10th Pass

Desirable Qualification: Experience in field work

Age: Maximum 28 years as on 11.11.2013. Age relaxation as per GOI rules.

Emoluments: Rsl2,040l Per Month

Terms: posts are purely temporary. Appointment will be initially made for a period of one (01) year and may be extended further based on the performance of the candidate up to completion of the project.

Application & Selection procedure: candidates have to appear in the walk-in-interview in person along with an application/CV on plain paper giving details of at educational qualificationq experience and submit photocopies of relevant documents at the time of interview. Selection will be based on the performance of the candidate in the interview' Candidates will not be sent any interview call letter separately. No TA/DA will be paid to the candidate for appearing in the interview. selection is not possible without appearing in the interview. All candidates must report by 9:00am on the date of interview. Advance copy of CV may be sent to m.sarathipartha@gmail.com

Advertisement: http://www.jalma-icmr.org.in/P_S_M_advertisment.pdf

NCBIBLAST+ 2.14.1 now available

Neel — Wed, 30 Aug 2023 02:36:13 -0500

#NCBIBLAST+ 2.14.1 now available with improved documentation, faster and more reliable database downloads, and some bug fixes.

Check out the changes they made.

They added the cleanup-blastdb-volumes.py script to remove unused BLAST database volumes. Read the documentation here.

They also switched the protocol from ftp to https to access BLAST databases for increased performance and reliability when downloading data from the NCBI with the update_blastdb.pl script.

And fixed a few bugs related to downloading data from the NCBI, and mt_mode crashing blastn and blastx.

Check out the release notes.

Download BLAST+ 2.14.1

Questions or comments? Please write the BLAST help desk.

More info and download: https://blast.ncbi.nlm.nih.gov/doc/blast-news/2023-BLAST-News.html