Awk is a programming language that is specifically designed for quickly manipulating space delimited data. Although you can achieve all its functionality with Perl, awk is simpler in many practical cases.

Why awk? You can replace a pipeline of 'stuff | grep | sed | cut...' with a single call to awk. For a simple script, most of the timelag is in loading these apps into memory, and it's much faster to do it all with one. This is ideal for something like an openbox pipe menu where you want to generate something on the fly. You can use awk to make a neat one-liner for some quick job in the terminal, or build an awk section into a shell script. You can find a lot of online tutorials, but here I will only show a few examples which cover most of bioinformatician daily uses of awk.

choose rows where column 3 is larger than column 5:

awk '$3>$5' input.txt > output.txt

extract column 2,4,5:

awk '{print $2,$4,$5}' input.txt > output.txt

awk 'BEGIN{OFS="\t"}{print $2,$4,$5}' input.txt

show rows between 20th and 80th:

awk 'NR>=20&&NR<=80' input.txt > output.txt

calculate the average of column 2:

awk '{x+=$2}END{print x/NR}' input.txt

regex (egrep):

awk '/^test[0-9]+/' input.txt

calculate the sum of column 2 and 3 and put it at the end of a row or replace the first column:

awk '{print $0,$2+$3}' input.txt

awk '{$1=$2+$3;print}' input.txt

join two files on column 1:

awk 'BEGIN{while((getline<"file1.txt")>0)l[$1]=$0}$1 in l{print $0"\t"l[$1]}' file2.txt > output.txt

count number of occurrence of column 2 (uniq -c):

awk '{l[$2]++}END{for (x in l) print x,l[x]}' input.txt

apply "uniq" on column 2, only printing the first occurrence (uniq):

awk '!($2 in l){print;l[$2]=1}' input.txt

count different words (wc):

awk '{for(i=1;i!=NF;++i)c[$i]++}END{for (x in c) print x,c[x]}' input.txt

deal with simple CSV:

awk -F, '{print $1,$2}'

substitution (sed is simpler in this case):

awk '{sub(/test/, "no", $0);print}' input.txt

OK now here's where to read this stuff properly explained. roll

Two thorough tutorials:

http://www.gnu.org/software/gawk/manual/gawk.html

http://www.grymoire.com/Unix/Awk.html

A famous list of useful one-liners - though they're short, many are quite tricky:

http://www.pement.org/awk/awk1line.txt

And some nice explanations of those one-liners. After reading this you'll have a pretty good grasp!

http://www.catonmat.net/blog/awk-one-li … -part-one/

http://www.catonmat.net/blog/ten-awk-ti … -pitfalls/

Adapter sequences:
Optimal enzymes for amplifying sequencing libraries.
Quail, M. a et al. Nat. Methods 9, 10-1 (2012).

GAGE:
GAGE: A critical evaluation of genome assemblies and assembly algorithms.
Salzberg, S. L. et al. Genome Res. 22, 557-67 (2012).

KMC:
Disk-based k-mer counting on a PC.
Deorowicz, S., Debudaj-Grabysz, A. & Grabowski, S. BMC Bioinformatics 14, 160 (2013).

Kraken:
Kraken: ultrafast metagenomic sequence classification using exact alignments.
Wood, D. E. & Salzberg, S. L. Genome Biol. 15, R46 (2014).

MUMmer:
Versatile and open software for comparing large genomes.
Kurtz, S. et al. Genome Biol. 5, R12 (2004).

R:
R: A language and environment for statistical computing.
R Core Team (2013). R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.

RATT:
RATT: Rapid Annotation Transfer Tool.
Otto, T. D., Dillon, G. P., Degrave, W. S. & Berriman, M. Nucleic Acids Res. 39, e57 (2011).

SAMtools:
The Sequence Alignment/Map format and SAMtools.
Li, H. et al. Bioinformatics 25, 2078-9 (2009).

Trimmomatic:
Trimmomatic: A flexible trimmer for Illumina Sequence Data.
Bolger, A. M., Lohse, M. & Usadel, B. Bioinformatics 1-7 (2014).

(Indian Council of Medical Research )
Jehangir Merwanji Street, Parel, Mumbai 400 012

Advertisement No. 1/NIRRH/Ph.D. 2013
Admission to Ph.D. Programme – 2013

National Institute for Research in Reproductive Health, Mumbai, a premier institute of the Indian Council of Medical Research, conducts basic, clinical and operational research in different areas of reproductive health. The thrust areas of research include: Fertility Regulation, Infertility and Reproductive Disorders, Reproductive Tract Infections, Maternal and Child Health, Osteoporosis, Genetic Disorders, Stem Cell Biology, Structural Biology, Bioinformatics and Reproductive Toxicology. Institute is affiliated to the University of Mumbai for the award of Ph.D. degree in Applied Biology, Biochemistry, Life Sciences and Biotechnology. The institute invites applications from young and bright students for enrollment in Ph.D. programme.

More at http://www.nirrh.res.in/announcements/phd_program_2013.htm

Postdoc position (m/w/d)

in the field of computational evolutionary microbiology is available
for an initially limited period of 36 months at the earliest possible
date. The weekly working time corresponds to 100% of full employment
(If the legal requirements under collective bargaining law are met, the
tariff grouping is carried out up to pay scale 13 TV-L. The obligation
to teach amounts to 4 hours.

The Genomic Microbiology Group research interests are focused on
microbial genome evolution with an emphasis on the study of lateral gene
transfer. In our research we use both computational and experimental
approaches (see www.uni-kiel.de/genomik). The position offers the
opportunity to develop an independent research profile within the group
research focus. The successful applicant is expected to be involved
in teaching of bioinformatics and molecular evolution, including the
development of teaching materials (lectures/exercises/short videos).

Your profile:
· Doctoral or PhD degree in Molecular Evolution, Bioinformatics or
related fields.
· Knowledge and experience in programming (e.g., Python) and
biostatistical analysis (e.g., with R or MatLab).
· Any of the following expertise is an advantage: the analysis of
genomic or transcriptomic data, phylogenetic reconstruction,
comparative genomics.
· Good oral and written communication skills in English.
· Ability to teach in German is an advantage (alternatively, an
indication to do so from the 2nd year on).
· Skills and motivation to communicate and interact with other
scientists.

The Christian-Albrechts-University sees itself as a modern and
cosmopolitan employer. We welcome your application regardless of your age,
gender, cultural and social background, religion, ideology, disability
or sexual identity. We promote equality of the sexes.

The Christian-Albrechts-University is committed to the employment of
people with disabilities. Preference will be given to applications from
severely handicapped persons and persons of equal standing, provided
they are suitable.

We expressly welcome applications from people with a migration background.

For enquiries regarding the position, teaching obligations and research
topic please contact Prof. Tal Dagan: tdagan@ifam.uni-kiel.de.

Applications should be submitted by email to Mrs. Haacks
(dhaacks@ifam.uni-kiel.de) as a single PDF and include: (1) a letter of
motivation (max 1 page, Arial 11, line spacing 1.15), (2) CV, (3) PhD
certificate. Please use 'GMG postdoc application - [your name]'
as a subject.

Please, refrain from sending us application photos.

Application deadline: August 31 2021 or until the position is
filled. Interviews will take place during September/October 2021. The
planned starting date for the position is flexible (but in 2021).

More at >> http://www.baumbachlab.net/

Address of the bookmark: https://biokit.readthedocs.io/en/latest/index.html

To cite BioNumbers please refer to: Milo et al. Nucl. Acids Res. (2010) 38: D750-D753. When using a specific entry from the database it is highly recommended that you also specify the BioNumbers 6 digit ID, e.g. "BNID 100986, Milo et al 2010". 

Address of the bookmark: http://bionumbers.hms.harvard.edu/

For use case 1 we obtained the following ENCODE and ROADMAP datasets https://www.encodeproject.org/files/ENCFF446WOD/@@download/ENCFF446WOD.bed.gz, https://www.encodeproject.org/files/ENCFF546PJU/@@download/ENCFF546PJU.bam, https://www.encodeproject.org/files/ENCFF059BEU/@@download/ENCFF059BEU.bam. Blacklisted regions were obtained from http://mitra.stanford.edu/kundaje/akundaje/release/blacklists/hg38-human/hg38.blacklist.bed.gz. The human genome version hg38 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz.

For use case 2 we used the set of narrowPeak files summarized in https://github.com/wkopp/janggu_usecases/tree/master/extra/urls.txt (archived version v1.0.1). The human genome version hg19 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz

For use case 3 we used the ENCODE datasets https://www.encodeproject.org/files/ENCFF591XCX/@@download/ENCFF591XCX.bam, https://www.encodeproject.org/files/ENCFF736LHE/@@download/ENCFF736LHE.bigWig, https://www.encodeproject.org/files/ENCFF177HHM/@@download/ENCFF177HHM.bam as we as the GENCODE annotation v29 from ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.annotation.gtf.gz.

Address of the bookmark: http://mitra.stanford.edu/

More at http://changlab.stanford.edu/index.html