BOL: Related items

Step-by-Step Guide to Running Genome Assembly

Abhi — Fri, 13 Dec 2024 11:35:55 -0600

Genome assembly is a critical process in bioinformatics, enabling the reconstruction of an organism's genome from short DNA sequence reads. Whether you’re working on a new microbial genome or a complex eukaryotic organism, this guide will walk you through the steps of genome assembly using state-of-the-art tools and best practices.

What is Genome Assembly?

Genome assembly involves piecing together short DNA sequence reads generated by sequencing platforms (e.g., Illumina, PacBio, Oxford Nanopore) into longer, contiguous sequences called contigs. This can be performed as:

De Novo Assembly: Without a reference genome.
Reference-Guided Assembly: Using a reference genome to guide the assembly process.

Step 1: Preparing Your Data

Before starting the assembly, ensure that your raw sequencing data is high quality.

Input Data
- Short Reads: Illumina sequencing generates short, accurate reads ideal for scaffolding.
- Long Reads: PacBio and Nanopore sequencing provide long reads for resolving repetitive regions.
Quality Control (QC)
Use tools like FastQC or MultiQC to assess the quality of your reads:

fastqc reads.fastq multiqc .

Look for issues like low-quality bases, adapter contamination, or overrepresented sequences.
Read Trimming and Filtering
Trim low-quality bases and adapters using Trimmomatic or Cutadapt:

trimmomatic PE reads_R1.fastq reads_R2.fastq trimmed_R1.fastq trimmed_R2.fastq \ ILLUMINACLIP:adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:20 MINLEN:36

Step 2: Choosing an Assembly Strategy

Select an assembly strategy based on your data type:

Short-Read Assemblers:
- SPAdes: Popular for microbial genomes.
- Velvet: Fast for smaller genomes.
Long-Read Assemblers:
- Canu: Ideal for long-read datasets.
- Flye: Versatile for small and large genomes.
Hybrid Assemblers:
- MaSuRCA: Combines short and long reads.
- Unicycler: Optimized for bacterial genomes.

Step 3: Running the Assembly

3.1. SPAdes (Short-Read Assembly)

SPAdes is an excellent choice for small genomes, such as bacteria.

spades.py -1 trimmed_R1.fastq -2 trimmed_R2.fastq -o spades_output

The output includes assembled contigs (contigs.fasta) and scaffolds (scaffolds.fasta).

3.2. Canu (Long-Read Assembly)

Canu is designed for high-error long reads from PacBio or Nanopore.

canu -p genome -d canu_output genomeSize=4.7m -nanopore-raw reads.fastq

The output will be in canu_output/genome.contigs.fasta.

3.3. Hybrid Assembly with Unicycler

Unicycler combines short and long reads for improved assemblies.

unicycler -1 trimmed_R1.fastq -2 trimmed_R2.fastq -l long_reads.fastq -o unicycler_output

Step 4: Assessing Assembly Quality

After assembly, evaluate its quality using the following tools:

QUAST
QUAST generates assembly statistics, such as N50, genome size, and GC content:

quast contigs.fasta -o quast_output
BUSCO
BUSCO checks genome completeness by identifying conserved genes:

busco -i contigs.fasta -o busco_output -l fungi_odb10 -m genome
Assembly Graph Visualization
Visualize assembly graphs with Bandage:

Bandage load assembly_graph.gfa

Step 5: Post-Assembly Steps

Polishing
Improve assembly accuracy using tools like Pilon (for short reads) or Racon (for long reads).

racon long_reads.fasta mapped_reads.sam contigs.fasta > polished_contigs.fasta
Scaffolding
Link contigs into scaffolds using tools like SSPACE or Opera-LG if required.
Annotation
Annotate the assembled genome using Prokka for prokaryotes or Maker for eukaryotes.

prokka --outdir annotation_output --prefix genome contigs.fasta

Step 6: Sharing and Archiving

Submit to Public Repositories
Share your assembly in databases like NCBI GenBank, ENA, or DDBJ.
Metadata Preparation
Include detailed metadata for your submission, such as organism name, sequencing platform, and coverage.

Best Practices

Always perform quality checks at each stage to ensure data integrity.
Use multiple tools to cross-validate results when working with complex genomes.
Document parameters and software versions for reproducibility.

Conclusion

Genome assembly is a powerful process that transforms raw sequencing data into a coherent representation of an organism’s genome. By following this step-by-step guide, you can successfully assemble genomes and uncover valuable biological insights. Whether you’re assembling a microbial genome or tackling the complexities of a eukaryotic genome, these tools and strategies will set you on the path to success.

Genomic architecture surrounding the fusion site of human chromosome 2

LEGE — Tue, 04 Mar 2025 12:26:29 -0600

The article "Genomic Structure and Evolution of the Ancestral Chromosome Fusion Site in 2q13–2q14.1 and Paralogous Regions on Other Human Chromosomes (https://pmc.ncbi.nlm.nih.gov/articles/PMC187548/)" explores the genomic architecture surrounding the fusion site of human chromosome 2. This fusion event is a key evolutionary marker distinguishing humans from other great apes, as humans have 46 chromosomes while chimpanzees, gorillas, and orangutans possess 48. The fusion occurred through an end-to-end joining of two ancestral chromosomes, which remain separate in nonhuman primates.

Key Findings:

Chromosomal Fusion and Its Molecular Signature:
- The fusion site is located at 2q13–2q14.1 and is characterized by degenerate telomeric sequences appearing interstitially, indicating the historical head-to-head joining of ancestral chromosomes.
- Despite being a signature of a past fusion event, these telomeric repeats are no longer functional and have undergone sequence degradation over time.
Extensive Duplications in the Surrounding Genomic Region:
- The study identifies large-scale segmental duplications flanking the fusion site, with several of these regions duplicated and scattered across multiple chromosomes.
- These duplications are predominantly located in subtelomeric and pericentromeric regions, suggesting their role in genomic instability and chromosomal evolution.
Paralogous Regions and Their Evolutionary Relationships:
- A 168-kilobase (kb) segment near the fusion site has 98%–99% sequence identity with three regions on chromosome 9 (9pter, 9p11.2, and 9q13).
- Another 67-kb region distal to the fusion site shows a high degree of homology to sequences in chromosome 22qter.
- Additionally, a 100-kb segment exhibits 96% sequence identity with a region in chromosome 2q11.2.
Comparative Genomics and Evolutionary Implications:
- By comparing the duplicated sequences and their arrangement in primates, the researchers traced the order of duplication events leading to their present distribution.
- The presence of specific repetitive elements within these duplicated segments serves as evolutionary markers that help infer their historical rearrangements.
- Some of these duplicated regions are associated with chromosomal inversion breakpoints, potentially contributing to evolutionary changes in primates.
- Recurrent structural rearrangements in these regions have been linked to human chromosomal disorders.

Conclusions and Implications:

The findings provide valuable insights into the structural evolution of human chromosome 2, which played a crucial role in human speciation.
Understanding these segmental duplications and their evolutionary trajectories sheds light on genomic instability, which may contribute to human genetic diseases.
The study highlights how large-scale chromosomal rearrangements, such as fusion and duplication, have influenced the evolutionary divergence of humans from other primates.

This research advances our understanding of human genome evolution and offers a foundation for studying the effects of structural variants in genetic disorders.

When Thousands of Bacterial Genomes Become One Giant Map

LEGE — Thu, 27 Aug 2026 10:56:47 -0500

Imagine trying to understand a city by studying just one house.

You might learn a lot about that house—the rooms, the doors, the furniture—but you would miss the bigger story: the streets, the neighborhoods, and all the ways the city changes from one place to another.

Something similar happens when scientists study bacterial genomes one at a time.

Over the past decade, researchers have collected thousands of bacterial genomes. These genomes contain an enormous amount of information about how bacteria survive, adapt, and evolve. But comparing thousands of individual genomes can quickly become a computational maze.

That is where PANORAMA enters the story.

Developed by Jérôme Arnoux and colleagues, PANORAMA is a computational tool designed to explore bacterial pangenomes—the complete collection of genetic possibilities found across a species or group of related organisms. Instead of looking at every genome as an isolated object, the approach represents their shared and variable genetic features as a graph.

Think of this graph as a giant subway map.

Some stations appear on almost every route. These represent genes that are highly conserved. Other stations appear only on certain routes, representing genes that some bacteria possess while others do not. By looking at the entire network, scientists can begin to see not just *what genes exist*, but how they are organized and how biological systems are distributed across populations.

The researchers put PANORAMA to the test using 941 genomes of Pseudomonas aeruginosa, a bacterium important in human health. They used the tool to investigate biological systems, including bacterial defense mechanisms against viruses known as bacteriophages. They then expanded the analysis to more than 6,000 genomes from four Enterobacteriaceae species.

The result is more than a faster way to process data.

PANORAMA allows researchers to ask a bigger question: What can an entire microbial species do, genetically speaking?

By comparing pangenomes, the researchers could identify systems shared between species as well as distinctive features. They could also find recurring genomic locations where genetic material is inserted—clues that may reveal common evolutionary processes.

And this is perhaps the most exciting part of the story.

Every bacterial genome is like a page in a huge evolutionary book. Until recently, reading thousands of those pages together was difficult. PANORAMA provides a way to turn those pages into a map, allowing scientists to see patterns that might disappear when each genome is studied separately.

The study, published in PLOS Computational Biology in July 2026, presents PANORAMA as a foundation for large-scale comparative pangenomics. The software and accompanying analysis resources are openly available, giving other researchers the opportunity to explore microbial diversity themselves.

So the story is not really about one bacterium or one genome.

It is about changing the way we look at life.

Instead of asking, “What is inside this genome?”, scientists can increasingly ask, “What is the full genetic landscape of this species—and how did it become this way?”

And sometimes, when you stop looking at one house and finally see the whole city, the most interesting discoveries are hiding in the streets between them.

Read more at https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1013856

The Tree That Learned to Read Genomes

BioStar — Fri, 11 Sep 2026 00:51:02 -0500

Imagine trying to reconstruct a family tree when many family photographs are missing, some names are unclear, and different relatives remember the past differently. This is similar to the challenge scientists face when trying to understand how organisms are related. Their family records are genomes, and the family tree is a phylogenetic tree. With modern sequencing technologies, scientists now have access to thousands of genomes, but comparing them can be difficult, especially when genomes are incomplete or poorly annotated.

A recent study by Webster and Chapman introduces Buscogeny (https://github.com/Jwebster89/Buscogeny), an open-source bioinformatics pipeline designed to make this process easier. The tool uses BUSCO (Benchmarking Universal Single-Copy Orthologs), which identifies conserved genes that are expected to occur as single copies in particular groups of organisms. These genes act like evolutionary landmarks, allowing researchers to compare genomes and investigate their relationships.

Buscogeny brings several steps of phylogenomic analysis into one workflow. It identifies BUSCO genes, extracts and aligns their sequences, removes unreliable regions, combines information from multiple genes, and uses the resulting data to construct a phylogenetic tree. It also provides quality-control information, helping researchers identify genomes that may be too incomplete for reliable analysis.

The authors demonstrate the usefulness of Buscogeny with bacterial and fungal genomes, including Alternaria. This is particularly valuable because fungal genomes can vary greatly in quality and annotation. Ultimately, Buscogeny helps transform massive amounts of genomic data into an understandable evolutionary story. Each DNA sequence becomes a clue, each conserved gene becomes a landmark, and together these clues reveal a picture of how life is connected.

More at https://link.springer.com/article/10.1007/s10123-025-00752-6?

Sequanix: a dynamic graphical interface for Snakemake workflows

BioStar — Wed, 12 Feb 2020 01:20:34 -0600

A Python library dedicated to NGS analysis (e.g., tools to visualise standard NGS formats).
A set of pipelines dedicated to NGS in the form of Snakefiles (Makefile-like with Python syntax based on snakemake framework) with more than 80 re-usable rules (see Rules).
Original tools to help in the creation of such pipelines including HTML reports.
Standalone applications:
1. sequana_coverage ease the extraction of genomic regions of interest and genome coverage information
2. sequana_taxonomy performs a quick taxonomy of your FastQ. This requires dedicated databases to be downloaded.
3. Sequanix: GUI for snakemake workflows, a GUI for Snakemake workflows (hence Sequana pipelines as well)
More at https://sequana.readthedocs.io/en/master/

Address of the bookmark: https://github.com/sequana/sequana

Search Shell Command History

Rahul Nayak — Thu, 12 Jun 2014 17:43:34 -0500

We use couple of hundreads of command in daily basis. Most of them are actually repeated several time. The question remain open how do I search old command history under bash shell and modify or reuse it?

Now a days almost all modern shell allows you to search command history if enabled by user. Use history command to display the history list with line numbers. Lines listed with with a * have been modified by user.

Shell history search command

Type history at a shell prompt:
$ history

It will display the list of all used commandline history with an serial number.

To search particular command, enter:
$ history | grep command-name
$ history | egrep -i 'scp|ssh|ftp'
Emacs Line-Edit Mode Command History Searching

To get previous command containing string, hit [CTRL]+[r] followed by search string:

(reverse-i-search):

To get previous command, hit [CTRL]+[p]. You can also use up arrow key.

CTRL-p

To get next command, hit [CTRL]+[n]. You can also use down arrow key.

CTRL-n

fc command

Apart from hostory command there are fc command to extract the command from history. The fc stands for either "find command" or "fix command.

For example list last 10 command, enter:
$ fc -l 10
To list commands 130 through 150, enter:
$ fc -l 130 150
To list all commands since the last command beginning with ssh, enter:
$ fc -l ssh
You can edit commands 1 through 5 using vi text editor, enter:
$ fc -e vi 1 5

Delete command history

The -c option causes the history list to be cleared by deleting all of the entries:
$ history -c

Public Databases for Bioinformatics !

Jit — Tue, 23 Mar 2021 05:32:15 -0500

https://www.nature.com/articles/s41467-020-17155-y

Server Infrastructure:

File Server:

dhara: Synology 3614 Storage Appliance
4 Core Xeon
108TB disk storage
10Gb ethernet to SCG3
Access atx: dhara:5000
Has btsync server (try it - its much better than dropbox)

Compute Servers:

nandi: Kundaje and Phi Server
24 intel cores
256GB RAM
500GB of SSD storage 
36TB RAID6 local storage
4 Intel Phi's (space for 4 more GPU's)


durga: Montgomery and sensitive data
24 intel cores
256GB RAM
500GB of SSD RAID0 storage 
60TB RAID6 local storage

mitra: Bassik and Web/DB Server
24 core
256GB RAM 
500GB of SSD RAID0 storage 
36TB RAID6 local storage

vayu: Kundaje GPU server
4 core
64GB RAM 
200GB of SSD storage 
8TB RAID10 local storage
4 Nvidia GTX 970 4GB GPUs

amold: Bickel and SGE server
32 AMD core
128GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

wotan: Bickel and SGE server
64 AMD core
256GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

Filesystem:

/users/$USER
default home directory
full backups nightly 
nfs mount to dhara
should store code, papers, and other highly processed data here

/mnt/data/
globally accessible data
should store common data here
e.g. genomes and indexes, annotations, ENCODE data  
if you dont want this to count towards your quote you must chown

/mnt/lab_data/$LAB/
lab accessible data
should store lab project data here 
e.g. ATAC-seq prediction data, enhancer prediction, motif calls

/srv/scratch/$USER
fast local storage
not backed up, but on raid and data will never be deleted
most analysis should be performed here

/srv/persistent/$USER
fast local storage
synced nightly, but not backed up
       ie if the hard drives fail or you delete something and notice 
       within 24 hours we can recover. Otherwise not. (vs home which is 
       properly backed up )  
intermediate analysis products that would be hard to recover should be stored here 
       e.g. stochastic analysis results that need to be kept so that paper 
       results can be reproduced

/srv/www/$LABNAME/
web accessible from mitra.stanford.edu
*NOT BACKED UP*

Some parallel programming patterns:

# gzip a bunch of files
parallel gzip -- *.FILESTOGZIP

# fork example in python:
(for more detailed examples look at 
 https://github.com/nboley/grit/ grit/lib/multiprocessing_utils.py)

import os
import time
import random

import multiprocessing

class ProcessSafeOPStream( object ):
    def __init__( self, writeable_obj ):
        self.writeable_obj = writeable_obj
        self.lock = multiprocessing.Lock()
        self.name = self.writeable_obj.name
        return
    
    def write( self, data ):
        self.lock.acquire()
        self.writeable_obj.write( data )
        self.writeable_obj.flush()
        self.lock.release()
        return
    
    def close( self ):
        self.writeable_obj.close()

def worker(queue, ofp):
    # Try without this
    random.seed()
    while True:
        i = queue.get()
        if i == 'FINISHED': return
        # simulate an expensive function
        x = random.random()
        time.sleep(x/10)
        print i, x
        ofp.write("%i\t%s\n" % (i, x))

NSIMS = 10000
NPROC = 25

# populate queue
todo = multiprocessing.Queue()
for i in xrange(NSIMS): todo.put(i)
for i in xrange(NPROC): todo.put('FINISHED')

ofp = ProcessSafeOPStream( open("output.txt", "w") )

pids = []
for i in xrange(NPROC):
    pid = os.fork()
    if pid == 0:
       worker(todo, ofp)
       os._exit(0)
    else:
       pids.append(pid)  

for pid in pids:
    os.waitpid(pid, 0)

ofp.close()

print "FINISHED"

For use case 1 we obtained the following ENCODE and ROADMAP datasets https://www.encodeproject.org/files/ENCFF446WOD/@@download/ENCFF446WOD.bed.gz, https://www.encodeproject.org/files/ENCFF546PJU/@@download/ENCFF546PJU.bam, https://www.encodeproject.org/files/ENCFF059BEU/@@download/ENCFF059BEU.bam. Blacklisted regions were obtained from http://mitra.stanford.edu/kundaje/akundaje/release/blacklists/hg38-human/hg38.blacklist.bed.gz. The human genome version hg38 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz.

For use case 2 we used the set of narrowPeak files summarized in https://github.com/wkopp/janggu_usecases/tree/master/extra/urls.txt (archived version v1.0.1). The human genome version hg19 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz

For use case 3 we used the ENCODE datasets https://www.encodeproject.org/files/ENCFF591XCX/@@download/ENCFF591XCX.bam, https://www.encodeproject.org/files/ENCFF736LHE/@@download/ENCFF736LHE.bigWig, https://www.encodeproject.org/files/ENCFF177HHM/@@download/ENCFF177HHM.bam as we as the GENCODE annotation v29 from ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.annotation.gtf.gz.

Address of the bookmark: http://mitra.stanford.edu/

Scripts for the analysis of HGT in genome sequence data.

Jit — Wed, 29 Nov 2017 16:44:10 -0600

Scripts for the analysis of HGT in genome sequence data

Address of the bookmark: https://github.com/reubwn/hgt

kSNP3.0: SNP detection and phylogenetic analysis of genomes without genome alignment or reference genome

Jit — Fri, 08 Dec 2017 16:48:40 -0600

Sept. 20, 2017 Version 3.1 released. Major upgrade. Version 3.1 fixes the problems with SNP annotation that arose when NCBI discontinued use of GI numbers. Please read carefully the Preface (page 3) and the File of annotated genomes section (pages 9-10) in the version 3.1 User Guide. Thanks to Tom Slezak for revsing the get_genbank_file3 script and to Tod Stuber (USDA) for testing version 3.1 even though he doesn't need the annotation feature. All users are encouraged to upgrade to version 3.1.

Address of the bookmark: https://sourceforge.net/projects/ksnp/files/

String graph based genome assembly software and tools !

Rahul Nayak — Tue, 19 Dec 2017 17:17:38 -0600

In graph theory, a string graph is an intersection graph of curves in the plane; each curve is called a "string". String graphs were first proposed by E. W. Myers in a 2005 publication. In recent Genome Research paper describing an innovative approach for assembling large genomes from NGS data caught our attention for several reasons. i) it give different "string graph" prospective of long lasting genome assembly problem ii) the paper is coauthored by Jared Simpson, the developer of ABySS assembler and Richard Durbin. iii) Simpson-Durbin algorithm is that it does not rely on de Bruijn graphs, and instead employs a different graph construction approach called ‘string graph’.

Following are the genome assembly tools based on string graph:

1.SGA (String Graph Assembler) https://github.com/jts/sga

Assembles large genomes from high coverage short read data. SGA is designed as a modular set of programs, which are used to form an assembly pipeline. SGA implements a set of assembly algorithms based on the FM-index. As the FM-index is a compressed data structure, the algorithms are very memory efficient. The SGA assembly has three distinct phases. The first phase corrects base calling errors in the reads. The second phase assembles contigs from the corrected reads. The third phase uses paired end and/or mate pair data to build scaffolds from the contigs. The output of this software is a PDF report that allows the properties of the genome and data quality to be visually explored. By providing more information to the user at the start of an assembly project, this software will help increase awareness of the factors that make a given assembly easy or difficult, assist in the selection of software and parameters and help to troubleshoot an assembly if it runs into problems.

2. SAGE: String-overlap Assembly of GEnomes https://github.com/lucian-ilie/SAGE2

SAGE, for de novo genome assembly. As opposed to most assemblers, which are de Bruijn graph based, SAGE uses the string-overlap graph. SAGE builds upon great existing work on string-overlap graph and maximum likelihood assembly, bringing an important number of new ideas, such as the efficient computation of the transitive reduction of the string overlap graph, the use of (generalized) edge multiplicity statistics for more accurate estimation of read copy counts, and the improved use of mate pairs and min-cost flow for supporting edge merging. The assemblies produced by SAGE for several short and medium-size genomes compared favourably with those of existing leading assemblers.

3. FSG: Fast String Graph

The new integrated assembler has been assessed on a standard benchmark, showing that fast string graph (FSG) is significantly faster than SGA while maintaining a moderate use of main memory, and showing practical advantages in running FSG on multiple threads. Moreover, we have studied the effect of coverage rates on the running times.

4. BASE https://github.com/dhlbh/BASE

It enhances the classic seed-extension approach by indexing the reads efficiently to generate adaptive seeds that have high probability to appear uniquely in the genome. Such seeds form the basis for BASE to build extension trees and then to use reverse validation to remove the branches based on read coverage and paired-end information, resulting in high-quality consensus sequences of reads sharing the seeds. Such consensus sequences are then extended to contigs. BASE is a practically efficient tool for constructing contig, with significant improvement in quality for long NGS reads. It is relatively easy to extend BASE to include scaffolding.

5. Fermi https://github.com/lh3/fermi/

Fermi is a de novo assembler with a particular focus on assembling Illumina short sequence reads from a mammal-sized genome. In addition to the role of a typical assembler, fermi also aims to preserve heterozygotes which are often collapsed by other assemblers. Its ultimate goal is to find a minimal set of unitigs to represent all the information in raw reads.

If you want to learn about String Graph assembler, please read the following papers -

i) The Fragment Assembly String Graph - E. W. Myers

This paper describes the String Graph concept.

ii) Efficient construction of an assembly string graph using the FM-index - Jared T. Simpson and Richard Durbin

This earlier paper from Simpson and Durbin

iii) Efficient de novo assembly of large genomes using compressed data structures - Jared T. Simpson and Richard Durbin