Next generation quantitative genetics in plants. Jiménez-Gómez, Frontiers in Plant Science 2:77, 2011 Full Text [equally relevant to animal and microbial systems]

Sense from sequence reads: methods for alignment and assembly. Flicek & Birney, Nat Methods 6(11 Suppl):S6-S12, 2009. Full Text

Library construction and experimental design

Statistical design and analysis of RNA sequencing data. Auer & Doerge, Genetics 185(2):405-16, 2010. PubMedCentral

Biases in Illumina transcriptome sequencing caused by random hexamer priming. Hansen et al., Nucleic Acids Res. 38(12): e131, 2010. PubMedCentral

Analyzing and minimizing PCR amplification bias in Illumina sequencing libraries. Aird et al, Genome Biology 12:R18, 2011 Full Text

Amplification-free Illumina sequencing-library preparation facilitates improved mapping and assembly of GC-biased genomes. Kozarewa et al, Nature Methods 6(4):291-5, 2009 PubMedCentral

Cost-effective, high-throughput DNA sequencing libraries for multiplexed target capture. Rohland & Reich, Genome Research 22(5): 939–946. PubMedCentral

Data formats, data management, and alignment software tools

The Sequence Alignment/Map format and SAMtools. Li et al, Bioinformatics 25(16):2078-9, 2009 PubMedCentral

SAM format specification file

Efficient storage of high throughput sequencing data using reference-based compression. Fritz et al, Genome Res 21(5):734-40, 2011. Full Text

Compression of DNA sequence reads in FASTQ format. Deorowicz & Grabowski, Bioinformatics 27(6):860-2, 2011. PubMed

Fast and accurate short read alignment with Burrows-Wheeler transform. Li & Durbin, Bioinformatics 25(14):1754-60, 2009. PubMedCentral

Improving SNP discovery by base alignment quality. Li H, Bioinformatics 27(8):1157-8, 2011. PubMed

BEDTools: a flexible suite of utilities for comparing genomic features. Quinlan and Hall, Bioinformatics 26:841-842, 2010. Publisher Website

Data quality assessment, filtering, and correction

SolexaQA: At-a-glance quality assessment of Illumina second-generation sequencing data. Cox et al, BMC Bioinformatics 11:485, 2010. PubMedCentral

TileQC: a system for tile-based quality control of Solexa data. Dolan & Denver, BMC Bioinformatics 9:250, 2008 PubMedCentral [requires a reference sequence]

Quake: quality-aware detection and correction of sequencing errors. Kelley et al, Genome Biol 11(11):R116, 2010. PubMed

FastQC: a quality control tool for high-throughput sequence data. Home Page

FASTX-toolkit: FASTQ/A short-reads pre-processing tools Home Page

Reference-free validation of short read data. Schröder et al, PLoS One 5(9):e12681, 2010. PubMedCentral

Correction of sequencing errors in a mixed set of reads. Salmela, Bioinformatics 26(10):1284, 2010. Full Text [includes error correction of SOLiD reads in colorspace]

Repeat-aware modeling and correction of short read errors. Yang et al, BMC Bioinformatics 12(Supp1):S52, 2011 PubMedCentral [requires a reference sequence]

HiTEC: accurate error correction in high-throughput sequencing data. Ilie et al, Bioinformatics 27(3):295, 2011 Full Text

Error correction of high-throughput sequencing datasets with non-uniform coverage. Medvedev et al., Bioinformatics 27(13):i137-41, 2011. PubMedCentral

De novo assembly

Velvet: algorithms for de novo short read assembly using de Bruijn graphs. Zerbino & Birney, Genome Res 18(5):821-9, 2008. u>PubMedCentral

Assembly of large genomes using second-generation sequencing. Schatz et al, Genome Res 20(9):1165-73, 2010. PubMedCentral

High-quality draft assemblies of mammalian genomes from massively parallel sequence data. Gnerre et al, PNAS 108(4): 1513-18, 2011 PubMedCentral

Genome assembly has a major impact on gene content: a comparison of annotation in two Bos taurus assemblies. Florea  et al., PLoS One 6(6):e21400, 2011. PubMedCentral

Artemis: an integrated platform for visualization and analysis of high-throughput sequence-based experimental data. Carver et al, Bioinformatics 28(4):464 - 469, 2012 PubMedCentral

Efficient de novo assembly of large genomes using compressed data structures. Simpson & Durbin, Genome Research 22:549-556, 2012 Full Text [Describes the String Graph Assembler (SGA), which assembled a human genome in less than 6 days using 54 Gb of RAM and a 123-processor compute cluster for calculation of an FM-index of the 1.2 billion reads]

Readjoiner: a fast and memory efficient string graph-based sequence assembler. Gonnella & Kurtz, BMC Bioinformatics 13: 82, 2012 PubMedCentral

Assemblathon 1: A competitive assessment of de novo short read assembly methods. Earl et al, Genome Research 21:2224-2241, 2011 Full Text

Chromatin immunoprecipation analysis: ChIP-seq

ChIP-seq: advantages and challenges of a maturing technology. Park, Nat Rev Genet. 10:669-80, 2009 PubMed

ChIP-seq and Beyond: new and improved methodologies to detect and characterize protein-DNA interactions. Furey, Nat Rev Genet 13: 840–852, 2012 Publisher Web Site

MuMoD: a Bayesian approach to detect multiple modes of protein–DNA binding from genome-wide ChIP data. Narlikar, Nucleic Acids Res 41:21–32, 2013 PubMed

Transcriptome analysis

Assembly and comparison to genome

Full-length transcriptome assembly from RNA-Seq data without a reference genome. Grabherr et al, Nature Biotechnology 29:644 - 652, 2011. PubMed [The software is called Trinity, and is available on Sourceforge.]

Comprehensive analysis of RNA-Seq data reveals extensive RNA editing in a human transcriptome. Peng et al, Nature Biotechnology 30:253 - 260, 2012. PubMed [Several comments on this paper question whether the reported differences are in fact evidence of editing or are simply sequencing errors - the authors stand by their conclusions, but the controversy demonstrates the importance of robust data analysis methods.]

Optimization of de novo transcriptome assembly from next-generation sequencing data. Surget-Groba & Montoya-Burgos, Genome Res 20(10):1432-40, 2010. Full Text

Rnnotator: an automated de novo transcriptome assembly pipeline from stranded RNA-Seq reads. Martin et al, BMC Genomics 11:663, 2010 Full Text

De novo assembly and analysis of RNA-seq data. Robertson et al, Nature Methods 7:909-912, 2010 Full Text [describes Trans-ABySS, a pipeline to use the ABySS parallel assembler for de novo transcriptome analysis]

Differential expression analysis

R-SAP: a multi-threading computational pipeline for the characterization of high-throughput RNA-sequencing data. Mittal & McDonald, Nucleic Acids Res, 2012 Full Text

Targeted RNA sequencing reveals the deep complexity of the human transcriptome. Mercer et al, Nature Biotechnology 30:99 - 104, 2012 Publisher Website

Differential gene and transcript expression analysis of RNA-Seq experiments with TopHat and Cufflinks. Trapnell et al, Nature Protocols 7:562 - 578, 2012 Publisher Website

Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling. Łabaj et al, Bioinformatics 27:i383 - i391, 2011 Full Text

Improving RNA-Seq expression estimates by correcting for fragment bias. Roberts et al, Genome Biol 12:R22, 2011 PubMed Central

Cloud-scale RNA-sequencing differential expression analysis with Myrna. Langmead et al, Genome Biol 11:R83, 2010 Full Text

From RNA-seq reads to differential expression results. Oshlack et al, Genome Biol 11(12):220, 2010 Full Text

DEGseq: an R package for identifying differentially expressed genes from RNA-seq data. Wang et al., Bioinformatics. 26(1):136-8. 2010 PubMed

DEseq: Differential expression analysis for sequence count data. Anders and Huber, Genome Biology 11:R106, 2010 Full Text

edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Robinson et al., Bioinformatics 26(1):139-40 2010 PubMedCentral

Two-stage Poisson model for testing RNA-seq data. Auer and Doerge, SAGMB 10(1), article 26 Full Text

Experimental design, preprocessing, normalization and differential expression analysis of small RNA sequencing experiments. McCormick et al., Silence2(1):2, 2011 PubMedCentral

RNA-Seq gene expression estimation with read mapping uncertainty. Li et al, Bioinformatics 26:493-500, 2010 PubMedCentral [describes the RSEM software package]

Comparing genomes and assemblies; variant detection

Versatile and open software for comparing large genomes. Kurtz et al, Genome Biol (5(2):R12, 2004. PubMedCentral [describes the MUMmer software for full-genome alignment & comparisons]

Searching for SNPs with cloud computing. Langmead et al, Genome Biol 10(11):R134, 2009 Full Text

Calling SNPs without a reference sequence. Ratan et al, BMC Bioinformatics 11:130, 2010 PubMedCentral

Microindel detection in short-read sequence data. Krawitz et al, Bioinformatics 26(6):722-9, 2010. Full Text

vipR: variant identification in pooled DNA using R. Altmann et al., Bioinformatics 27: i77-i84, 2011. PubMedCentral

Geoseq: a tool for dissecting deep-sequencing datasets. Gurtowski et al, BMC Bioinformatics 11:506, 2010. PubMedCentral [Geoseq is a web service that allows searching deep sequencing datasets with a reference sequence of a gene of interest]

Detecting and annotating genetic variations using the HugeSeq pipeline. Lam et al, Nature Biotechnology 30:226 - 229, 2012 Publisher Website, Home Page

Genome-wide LORE1 retrotransposon mutagenesis and high-throughput insertion detection in Lotus japonicus. Urbański et al, Plant J 64:731-741, 2012. Publisher Website [This paper describes a 2-dimensional pooling strategy with barcoding to allow use of Illumina sequencing to screen for retrotransposon insertion mutations, and includes a software package called FSTpoolit for analysis of the resulting sequence reads.]

Genotyping by sequencing

Genome-wide genetic marker discovery and genotyping using next-generation sequencing. Davey et al., Nat Rev Genet 12(7):499-510, 2011 PubMed [A review of methods available at the time]

A robust, simple genotyping-by-sequencing (GBS) approach for high diversity species. Elshire et al., PLoS One 6(5):e19379, 2011. Full Text

Development of high-density genetic maps for barley and wheat using a novel two-enzyme genotyping-by-sequencing approach. Poland et al., PLoS One 7(2): e32253, 2012. Full Text

Double digest RADseq: an inexpensive method for de novo SNP discovery and genotyping in model and non-model species. Peterson et al, PLoS One 7(5):e37135, . 2012. Full Text

Imputation of unordered markers and the impact on genomic selection accuracy. Rutkowski et al, G3 3(3):427-39, 2013. Full Text

Diversity Arrays Technology (DArT) and next-generation sequencing combined: genome-wide, high-throughput, highly informative genotyping for molecular breeding of Eucalyptus. Sansaloni et al., BMC Proceedings 5(Suppl 7):P54, 2011 Full Text

High-throughput genotyping by whole-genome resequencing. Huang et al., Genome Res 19(6):1068-76, 2009. Full Text

Multiplexed shotgun genotyping for rapid and efficient genetic mapping. Andolfatto et al. Genome Res 21(4):610-7, 2011. Full Text

Restriction-site Associated DNA (RAD) markers

Rapid SNP discovery and genetic mapping using sequenced RAD markers. Baird et al, PLoS One 3(10):e3376, 2008 Full Text

Linkage mapping and comparative genomics using next-generation RAD sequencing of a non-model organism. Baxter et al., PLoS One 6(4):e19315, 2011. Full Text

Genome evolution and meiotic maps by massively parallel DNA sequencing: spotted gar, an outgroup for the teleost genome duplication. Amores et al, Genetics 188(4):799-808, 2011. PubMed

Construction and application for QTL analysis of a Restriction-site Associated DNA (RAD) linkage map in barley. Chutimanitsakun et al, BMC Genomics 4; 12:4, 2011. Full Text

RAD tag sequencing as a source of SNP markers in Cynara cardunculus L. Scaglione et al., BMC Genomics 13:3, 2012. Full Text

Paired-end RAD-seq for de novo assembly and marker design without available reference. Willing et al., Bioinformatics 27(16):2187-93, 2011. Publisher Website

Local de novo assembly of RAD paired-end contigs using short sequencing reads. Etter et al., PLOS ONE 6(4): e18561, 2011. Full Text

Stacks: building and genotyping loci de novo from short-read sequences. Catchen et al., G3: Genes, Genomes, Genetics, 1:171-182, 2011. Full Text, Home Page

Rainbow: an integrated tool for efficient clustering and assembling RAD-seq reads. Chong et al, Bioinformatics 28(21):2732-7, 2012. Publisher Website

UK RAD Sequencing Wiki page, with bibliography and RADTools software download Home Page

Workspace environments

Papers

Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Goecks et al, Genome Biol 11(8):R86, 2010 PubMedCentral

Galaxy Cloudman: Delivering compute clusters. BMC Bioinformatics 11(Suppl. 12):S4, 2010 Full Text

The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. McKenna et al, Genome Res 20(9):1297-303, 2010. PubMedCentral

A framework for variation discovery and genotyping using next-generation DNA sequencing data. DePristo et al., Nat Genet 43(5):491-8, 2011. PubMed

Online resources

The R statistical computing environment includes Bioconductor, a specialized set of tools for analysis of microarray and high-throughput sequencing data. Introductory materials from on-line or short workshops are widely available online; examples are Evomics2012 Bioconductor-tutorial.pdf, and Intro to Bioconductor. Materials from an advanced course on high-throughput genetic data analysis are at Seattle 2012 materials. Thomas Girke of UC-Riverside has written a very complete set of manuals describing the use of R and Bioconductor for analysis of genomic datasets, available at R and Bioconductor Manuals.
Manuals and contributed documentation for R are available at the R-project.org website, and video tutorials are also available on Youtube; those posted by Tutorlol are brief, clear, and to the point.
Materials from a series of mini-courses in R taught in 2010 at UCLA are available:

• Intro to programming and graphics
• Data manipulation and functions
• Graphics for exploratory data analysis
• Introductory statistics
• Linear regression

A Little Book of R for Bioinformatics is an on-line resource with information and exercises to provide practice in bioinformatics analysis of DNA sequences and other biological data in R.
Many books on specific topics in R programming are also available through Amazon or other vendors.

Cloud computing resources

The case for cloud computing in genome informatics. Lincoln Stein, Genome Biol. 11(5):207, 2010 Pubmed

Galaxy Cloudman: delivering cloud compute clusters. Afgan et al, BMC Bioinformatics 11(Suppl 12):S4, 2010 Full Text

CloudBioLinux is an open-source project that provides a bioinformatics Linux system for cloud computing, pre-configured with a variety of software tools installed and ready to use.

A tutorial on getting started with CloudBioLinux on the Amazon Web Services Elastic Compute Cloud (EC2)

“Deploying Galaxy on the Cloud” – slides from a presentation by Enis Afgan (Emory University) at the
 Bioinformatics Open Source Conference in Boston, July 2010

A screencast that provides a step-by-step guide to starting a Galaxy cluster in the EC2 environment

A webpage that has the same information in text form, and is the basis for the screencast

The iPlant Collaborative, an NSF-funded project to create computational resources for plant biology research, provides access to cloud computing resources through Atmosphere

SeqWare Query Engine: storing and searching sequence data in the cloud. O’Connor et al, BMC Bioinformatics 11(Suppl 12):S2, 2010 Full Text

An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics. Taylor, BMC Bioinformatics 11(Suppl 12):S1, 2010 Full Text

Links to Linux command-line tutorials and resources

Tutorials for AWK, a powerful tool for handling data tables

• A set of awk notes from Boston University
• Bruce Barnett's awk tutorial
• Greg Goebel's awk tutorial
• Executing an awk command from R to simplify data exploratory analysis, from Lex Nederbragt

Tutorials for bash shell scripting

• A tutorial at linuxconfig.org
• A Getting Started With Bash tutorial at hypexr.org
• Mendel Cooper's Advanced Bash Shell-Scripting Guide

Tutorials for sed, the command-line stream editor

• A tutorial at Rutgers
• Peteris Krumins claims to have the World's Best Introduction to Sed; take a look and judge for yourself.
• Bruce Barnett's sed tutorial.

Links to other useful sites

The SEQanswers online community has forums on several topics related to sequencing; the bioinformatics forum is the most active.

The SEQanswers Software Wiki is a list of software for analysis of sequencing data

Biostar is another online community for questions and answers on bioinformatics and computational genomics.

Information on file formats used by the University of California - Santa Cruz Genome Browser is on the FAQ list

A manual for the Integrated Genome Browser visualization tool is here

Course materials for a short course entitled Introduction to R and Bioconductor, held in Seattle in Dec 2010

Genomic Regions Enrichment of Annotations Tool - A web service to test for over-representation of specific ontology categories among genes near ChIP-seq peaks

Next-gen-seq software - a list of software packages, both commercial and open-source, related to analysis of deep sequencing datasets

Software from the Center for Bioinformatics and Computational Biology, University of Maryland - many useful programs, all open-source

PLAZA: a comparative genomics resource to study gene and genome evolution in plants; described by Proost et al, Plant Cell 21:3718, 2010 Full Text

The European Bioinformatics Institute provides tools ArrayExpressHTS and R-Cloud for analysis of transcriptome data

1. RNA-Seq Analysis Pipelines

RNA-seq is one of the most popular techniques for studying RNA. These tools streamline processing raw sequence data:

• FASTQC: For quality control of raw RNA-seq reads.
• Trimmomatic: For trimming and filtering RNA-seq reads.
• HISAT2/STAR: High-performance aligners for RNA-seq reads.
• FeatureCounts: For quantifying gene expression.
• DESeq2/EdgeR: For differential expression analysis.

2. Transcriptome Assembly and Annotation

For analyzing transcriptomes from non-model organisms or assembling novel transcripts:

• Trinity: For de novo transcriptome assembly.
• StringTie: For transcript assembly and quantification from RNA-seq alignments.
• TransDecoder: To predict coding regions within assembled transcripts.
• TAU: Tools for annotating non-coding and coding RNAs.

3. Exploring Non-Coding RNA (ncRNA)

Non-coding RNAs play critical regulatory roles. Dedicated tools for studying them include:

• Infernal: For identifying ncRNA sequences based on covariance models.
• Rfam: Database and tools for ncRNA families.
• miRDeep: For identifying microRNAs in RNA-seq datasets.

4. RNA Structure and Motif Analysis

Structural biology of RNA helps in understanding its function:

• RNAfold (ViennaRNA): Predicts secondary structures from RNA sequences.
• RNAstructure: Tools for RNA secondary structure prediction and analysis.
• MEME Suite: For identifying motifs in RNA sequences.
• IntaRNA: For RNA-RNA interaction prediction.

5. RNA Editing and Modifications

Epitranscriptomics is a growing field focusing on RNA modifications:

• REDItools: For RNA editing analysis.
• m6Aboost: For identifying m6A modifications in RNA.

6. Long-Read RNA Sequencing Analysis

Long-read technologies like Nanopore and PacBio are transforming RNA research:

• FLAIR: For isoform-level analysis of long-read RNA-seq data.
• NanoMod: For detecting modifications in RNA from Nanopore sequencing.

7. RNA-Protein Interactions

To study RNA-protein interactions and complexes:

• RBPmap: For identifying RNA-binding protein motifs.
• PARalyzer: For analyzing PAR-CLIP data.

8. Functional Enrichment Analysis

Understanding biological functions and pathways from RNA-seq data:

• getENRICH: A tool designed for pathway enrichment analysis of non-model organisms (hypergeometric P-value calculation with FDR correction).
• ClusterProfiler: For GO and KEGG pathway enrichment analysis.

9. Visualization and Data Sharing

Presenting and sharing RNA sequence analysis results effectively:

• IGV: Genome browser for visualizing RNA-seq alignments.
• Circos: Circular visualization of RNA-seq data.
• DashBio: A Python library for creating bioinformatics visualizations.

Conclusion

The bioinformatics landscape for RNA sequence analysis is vast, with tools catering to specific needs. Whether you’re studying coding RNAs, non-coding RNAs, or exploring RNA-protein interactions, the right tools can transform your data into biological insights.

Address of the bookmark: http://www.rdatamining.com/

While the expectations are not entirely off base, the reality of life as a bioinformatician is a mix of exciting discoveries, troubleshooting, and, let’s admit it, a fair amount of frustration. Here’s what it’s really like:

1. Expectation: Seamlessly Working with Perfect Datasets

Reality: You often receive messy, incomplete, or poorly annotated datasets. Hours are spent cleaning, normalizing, and validating data before you even begin your analysis. "Garbage in, garbage out" is a constant reminder in your workflow. Tools designed to handle these problems exist, but they require significant customization, which adds another layer of complexity.

2. Expectation: Effortless Multidisciplinary Integration

Reality: Bridging biology and computational science is far from straightforward. You need to be proficient in both domains while keeping up with advancements in genomics, machine learning, and statistics. Additionally, collaborating with biologists who might not be fluent in computational jargon requires patience and effective communication skills.

3. Expectation: Rapid, Groundbreaking Results

Reality: Analysis often involves waiting—waiting for scripts to run, pipelines to complete, or software to install. Bioinformatics projects are iterative; you analyze, debug, and refine repeatedly. A single project might take months to complete due to unforeseen challenges, like computational bottlenecks or the need for additional experiments.

4. Expectation: Beautiful Visualizations with a Click

Reality: While tools like R, Python, and specialized software can create stunning plots, generating a publication-ready visualization requires significant effort. You’ll spend hours tweaking axes, labels, and color palettes, ensuring clarity and accuracy.

5. Expectation: All Work, No Bugs

Reality: Debugging is an integral part of the job. Whether it’s a misconfigured server, a script throwing unexpected errors, or a pipeline breaking due to an update, you’ll develop a knack for problem-solving under pressure.

6. Expectation: Ample Time for Skill Development

Reality: Bioinformatics moves fast. Juggling ongoing projects, tight deadlines, and the constant stream of new tools and algorithms leaves little time for leisurely learning. Staying updated requires proactive effort—evenings, weekends, or dedicated study breaks.

7. Expectation: Publishing Papers Regularly

Reality: Publishing in bioinformatics is a marathon, not a sprint. Your analysis needs to be thorough, reproducible, and supported by strong biological insights. Reviewers often demand additional experiments or clarifications, stretching the timeline even further.

8. Expectation: A Clear Career Path

Reality: Bioinformatics offers diverse career paths, from academia and industry to healthcare and government. However, the choice can be daunting, with each path requiring unique skill sets and presenting different challenges. Navigating these options takes time, research, and sometimes trial and error.

Finding Joy in the Chaos

Despite these challenges, being a bioinformatician is immensely rewarding. You are at the forefront of science, enabling discoveries that impact medicine, agriculture, and the environment. The thrill of uncovering insights hidden in complex datasets and the satisfaction of solving biological puzzles make the hard work worthwhile.

Advice for Aspiring Bioinformaticians

• Embrace Learning: The field is ever-evolving. Stay curious and adaptable.
• Develop Communication Skills: Bridging the gap between biology and computation is as much about explaining your methods as it is about applying them.
• Find a Community: Collaborate with peers, join forums, and attend conferences to stay inspired and updated.
• Celebrate Small Wins: Every cleaned dataset, successful script, or informative plot is a step forward.

Bioinformatics is a blend of science, technology, and artistry. While the reality might not match the polished expectations, the journey is nothing short of exhilarating. If you’re ready to embrace the chaos and keep learning, the field of bioinformatics will never cease to amaze you.

Awk is a programming language that is specifically designed for quickly manipulating space delimited data. Although you can achieve all its functionality with Perl, awk is simpler in many practical cases.

Why awk? You can replace a pipeline of 'stuff | grep | sed | cut...' with a single call to awk. For a simple script, most of the timelag is in loading these apps into memory, and it's much faster to do it all with one. This is ideal for something like an openbox pipe menu where you want to generate something on the fly. You can use awk to make a neat one-liner for some quick job in the terminal, or build an awk section into a shell script. You can find a lot of online tutorials, but here I will only show a few examples which cover most of bioinformatician daily uses of awk.

choose rows where column 3 is larger than column 5:

awk '$3>$5' input.txt > output.txt

extract column 2,4,5:

awk '{print $2,$4,$5}' input.txt > output.txt

awk 'BEGIN{OFS="\t"}{print $2,$4,$5}' input.txt

show rows between 20th and 80th:

awk 'NR>=20&&NR<=80' input.txt > output.txt

calculate the average of column 2:

awk '{x+=$2}END{print x/NR}' input.txt

regex (egrep):

awk '/^test[0-9]+/' input.txt

calculate the sum of column 2 and 3 and put it at the end of a row or replace the first column:

awk '{print $0,$2+$3}' input.txt

awk '{$1=$2+$3;print}' input.txt

join two files on column 1:

awk 'BEGIN{while((getline<"file1.txt")>0)l[$1]=$0}$1 in l{print $0"\t"l[$1]}' file2.txt > output.txt

count number of occurrence of column 2 (uniq -c):

awk '{l[$2]++}END{for (x in l) print x,l[x]}' input.txt

apply "uniq" on column 2, only printing the first occurrence (uniq):

awk '!($2 in l){print;l[$2]=1}' input.txt

count different words (wc):

awk '{for(i=1;i!=NF;++i)c[$i]++}END{for (x in c) print x,c[x]}' input.txt

deal with simple CSV:

awk -F, '{print $1,$2}'

substitution (sed is simpler in this case):

awk '{sub(/test/, "no", $0);print}' input.txt

OK now here's where to read this stuff properly explained. roll

Two thorough tutorials:

http://www.gnu.org/software/gawk/manual/gawk.html

http://www.grymoire.com/Unix/Awk.html

A famous list of useful one-liners - though they're short, many are quite tricky:

http://www.pement.org/awk/awk1line.txt

And some nice explanations of those one-liners. After reading this you'll have a pretty good grasp!

http://www.catonmat.net/blog/awk-one-li … -part-one/

http://www.catonmat.net/blog/ten-awk-ti … -pitfalls/

Why Remote Session Management is a Must-Have

In bioinformatics, tasks like genome assembly, RNA-seq analyses, and phylogenetic computations often take hours or days. A dropped SSH connection can result in:

• Lost Progress: Restarting a job from scratch wastes valuable time.
• Workflow Interruptions: Disruptions can derail your focus and productivity.
• Corrupted Data: Interrupted processes may lead to incomplete or corrupted outputs.

By integrating screen, tmux, or mosh into your workflow, you can avoid these setbacks and ensure a seamless experience.

Screen: The Classic Workhorse

Screen is a terminal multiplexer that comes pre-installed on most Linux systems. It allows you to manage multiple terminal sessions and reconnect to them even after being disconnected.

Getting Started with Screen:

1. Start a Session:
   screen
2. Detach from a Session:
   Press Ctrl+A, then D.
3. Reattach to a Session:
   screen -r

Pro Tip: Enhance your screen experience with a customized .screenrc configuration file. Download one here: Get .screenrc.

Tmux: A Modern Alternative

Tmux takes everything great about screen and adds modern features, including better key bindings and intuitive session management. It\u2019s perfect for bioinformaticians who want more control over their workflow.

Getting Started with Tmux:

1. Start a Session:
   tmux
2. Detach from a Session:
   Press Ctrl+B, then D.
3. Reattach to a Session:
   tmux attach

Customize Your Tmux Experience:
Use a .tmux.conf file to personalize your setup. Grab one here: Download .tmux.conf.

Mosh: The Mobile Shell for Unreliable Connections

SSH works well for stable networks, but it struggles in areas with spotty connectivity. Enter Mosh, the Mobile Shell. Designed for intermittent networks, Mosh keeps your session alive even when the connection drops temporarily.

Why Mosh is a Game-Changer:

• No lag over high-latency networks.
• Automatically reconnects when the network is restored.
• Ideal for working on the go, from cafes to trains.

Getting Started with Mosh:

1. Install Mosh:
   sudo apt install mosh # For Debian/Ubuntu
2. Connect to a Server:
   mosh username@server

Learn more at mosh.org.

Why This Matters for Bioinformatics

Every bioinformatician knows the value of time and data integrity. Tools like screen, tmux, and mosh provide a lifeline when running long analyses, enabling you to:

• Safeguard your work against disconnections.
• Easily manage multiple workflows in parallel.
• Stay productive, even in challenging environments.

Quickstart Cheat Sheet

• Screen:

  screen # Start a session Ctrl+A, D # Detach screen -r # Reattach

• Tmux:

  tmux # Start a session Ctrl+B, D # Detach tmux attach # Reattach

• Mosh:

  mosh username@server

Final Thoughts

As a bioinformatician, your time is too valuable to spend restarting analyses due to technical hiccups. With screen, tmux, and mosh in your toolkit, you can work smarter, protect your progress, and stay productive no matter where you are. Start using these tools today and transform the way you work with remote systems.

Let me know how these tools work for you, and don\u2019t forget to follow for more bioinformatics tips!

(Indian Council of Medical Research )
Jehangir Merwanji Street, Parel, Mumbai 400 012

Advertisement No. 1/NIRRH/Ph.D. 2013
Admission to Ph.D. Programme – 2013

National Institute for Research in Reproductive Health, Mumbai, a premier institute of the Indian Council of Medical Research, conducts basic, clinical and operational research in different areas of reproductive health. The thrust areas of research include: Fertility Regulation, Infertility and Reproductive Disorders, Reproductive Tract Infections, Maternal and Child Health, Osteoporosis, Genetic Disorders, Stem Cell Biology, Structural Biology, Bioinformatics and Reproductive Toxicology. Institute is affiliated to the University of Mumbai for the award of Ph.D. degree in Applied Biology, Biochemistry, Life Sciences and Biotechnology. The institute invites applications from young and bright students for enrollment in Ph.D. programme.

More at http://www.nirrh.res.in/announcements/phd_program_2013.htm

Bioinformatics is a rapidly growing field at the intersection of biology, computer science, mathematics, and statistics. As data volumes increase, as well as the diversity of data types (genomics, proteomics, metabolomics, imaging, single‑cell data, etc.), the need for robust computational methods, rigorous models, and reproducible tools has never been greater.

A key decision for researchers is: Where should I publish my work? The choice of journal impacts visibility, peer recognition, and long‑term influence of your research. Below I provide a guide to leading journals in bioinformatics, criteria for selecting the journal that best fits your work, and why these considerations matter.

Leading Journals in Bioinformatics

Criteria to Use When Choosing a Journal

• Scope & Audience
• Impact & Visibility
• Review Time & Speed
• Open Access
• Cost / APCs
• Reputation vs Practical Fit
• Reproducibility, Data & Code Sharing Policies
• Indexing & Reach
• Quality of the field
• Accelerating discovery
• Fair access
• Credibility & trust
• Read recent papers in the journal
• Tailor the manuscript
• Check the author guidelines
• Have backup journals ready
• More emphasis on machine learning / AI
• Single‑cell, spatial omics, multimodal data
• Cloud workflows, reproducible pipelines
• Preprints / open peer review
• Alternative metrics (software use, downloads, community adoption)

Selecting where to publish in bioinformatics isn’t just about prestige; it’s about reaching the right audience, ensuring your work is usable, and contributing to the field responsibly.

Main mission:

Develop and implement strategies for whole-genome sequencing of non-model
species
Generate high-quality de novo genome assemblies using short- and long-read
sequencing technologies
Perform genome annotation and structural/functional characterization
Conduct comparative genomic analyses across related species or populations
Design and implement genome-wide association studies (GWAS) to identify
loci associated with phenotypic or adaptive traits
Integrate genomic, phenotypic, and environmental datasets
Contribute to the development of reproducible bioinformatics pipelines

Activités (Activities):

Lead the genomic component of the research project
High-molecular-weight DNA extraction optimization
Long-read genome assembly (PacBio HiFi / ONT)
Genome polishing and quality assessment (BUSCO, QUAST)
Structural and functional annotation
Variant discovery (SNPs, indels, SVs)
Population genomic analyses (FST, demographic inference)
Mixed-model GWAS accounting for structure
Workflow development (Snakemake/Nextflow)
HPC-based pipeline implementation
Publish results in peer-reviewed journals
Present findings at international conferences
Collaborate with experimental and computational team members
Contribute to project development
Mentor graduate students when appropriate

More at https://evol.mcmaster.ca/brian/evoldir/PostDocs//MontpellierU.ComparativeGenomics

To cite BioNumbers please refer to: Milo et al. Nucl. Acids Res. (2010) 38: D750-D753. When using a specific entry from the database it is highly recommended that you also specify the BioNumbers 6 digit ID, e.g. "BNID 100986, Milo et al 2010". 

Address of the bookmark: http://bionumbers.hms.harvard.edu/