https://bioinformaticsonline.com/snippets?offset=310 https://bioinformaticsonline.com/snippets/view/37516/perl-script-to-extract-sequence-by-ids-from-multifasta-file Thu, 09 Aug 2018 10:39:27 -0500 https://bioinformaticsonline.com/snippets/view/37516/perl-script-to-extract-sequence-by-ids-from-multifasta-file #!/usr/bin/perl -w use strict; my $idsfile = "$ARGV[0]"; my $seqfile = "$ARGV[1]"; my %ids = (); open FILE, $idsfile; while(<FILE>) { chomp; $ids{$_} += 1; } close FILE; local $/ = "\n>"; # read by FASTA record open FASTA, $seqfile; while (<FASTA>) { chomp; my $seq = $_; my ($id) = $seq =~ /^>*(\S+)/; # parse ID as first word in FASTA header if (exists($ids{$id})) { $seq =~ s/^>*.+\n//; # remove FASTA header $seq =~ s/\n//g; # remove endlines print ">$id\n$seq\n"; } } close FASTA;]]> Abhimanyu Singh https://bioinformaticsonline.com/snippets/view/37505/install-python-locally-on-shared-linux-server Tue, 07 Aug 2018 06:13:25 -0500 https://bioinformaticsonline.com/snippets/view/37505/install-python-locally-on-shared-linux-server #Install python to local directory #Firstly, I create a folder in my home directory, download the python source and extract it mkdir ~/python cd ~/python wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz tar zxfv Python-2.7.11.tgz find ~/python -type d | xargs chmod 0755 cd Python-2.7.11 #Then I compiled the source following its guideline ./configure --prefix=$HOME/python make && make install #Notice the prefix option, it is mandatory for this to work. The value of prefix option is to specify where to put the #related output of make command, by default it is in the /usr/local/ and we don't want that so we use our own #customized directory. #Here comes another important step. By the default, if we type python command, it will use the default python of #the system. We are going to update the environment variables to force the shell to use our new python. Edit #~/.bashrc_profile and add the following lines: export PATH=$HOME/python/Python-2.7.11/:$PATH export PYTHONPATH=$HOME/python/Python-2.7.11 #Finally, refresh the current session by running the command: source ~/.bashrc_profile #You might need to logout and login again for the environment to update properly. At this point, you should be #able to see a new python. To check, run this command: which python #it should show you the path to the python binary file, which is located in your home directory: ~/python/Python-#2.7.11/python Install pip #Pip is a program used to help us easily install python packages After installing python locally as described in #the first step, it is very easy to install pip. #Run the following command to install pip as a local user wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py -O - | python - --user #After finishing the installation, we need to update our PATH variable. Open ~/.bashrc_profile and add the #following line: export PATH=$HOME/.local/bin:$PATH #Again, reload the session by the command source ~/.bashrc_profile or logout and login. Then, check if pip #command is available: which pip #It should show a path pointing to your local directory: ~/.local/bin]]> Jit https://bioinformaticsonline.com/snippets/view/37500/installing-nextsv-on-ubuntu Mon, 06 Aug 2018 17:35:56 -0500 https://bioinformaticsonline.com/snippets/view/37500/installing-nextsv-on-ubuntu ➜ Tools git:(master) ✗ git clone https://github.com/Nextomics/nextsv.git Cloning into 'nextsv'... remote: Counting objects: 369, done. remote: Total 369 (delta 0), reused 0 (delta 0), pack-reused 369 Receiving objects: 100% (369/369), 51.81 MiB | 21.72 MiB/s, done. Resolving deltas: 100% (223/223), done. Checking connectivity... done. ➜ Tools git:(master) ✗ cd nextsv ➜ nextsv git:(master) pip install argparse Collecting argparse Downloading https://files.pythonhosted.org/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl Installing collected packages: argparse Successfully installed argparse-1.4.0 ➜ nextsv git:(master) pip install pysam Requirement already satisfied: pysam in /home/urbe/anaconda3/lib/python3.6/site-packages (0.14.1) ➜ nextsv git:(master) pip install networkx Requirement already satisfied: networkx in /home/urbe/anaconda3/lib/python3.6/site-packages (2.1) Requirement already satisfied: decorator>=4.1.0 in /home/urbe/anaconda3/lib/python3.6/site-packages (from networkx) (4.3.0) ➜ nextsv git:(master) pip install h5py Requirement already satisfied: h5py in /home/urbe/anaconda3/lib/python3.6/site-packages (2.7.1) Requirement already satisfied: numpy>=1.7 in /home/urbe/anaconda3/lib/python3.6/site-packages (from h5py) (1.14.3) Requirement already satisfied: six in /home/urbe/anaconda3/lib/python3.6/site-packages (from h5py) (1.11.0) ➜ nextsv git:(master) pip install PyIntervalTree Collecting PyIntervalTree Downloading https://files.pythonhosted.org/packages/aa/53/dbdb521c8e36301108114878ff1620255c559c12ba8fe0ce2eaf9b65cbef/PyIntervalTree-0.5.zip Requirement already satisfied: sortedcontainers in /home/urbe/anaconda3/lib/python3.6/site-packages (from PyIntervalTree) (1.5.10) Building wheels for collected packages: PyIntervalTree Running setup.py bdist_wheel for PyIntervalTree ... done Stored in directory: /home/urbe/.cache/pip/wheels/23/26/eb/83ac050fa1d31ed660d26a2508a47799728b29a9a0048a6b17 Successfully built PyIntervalTree Installing collected packages: PyIntervalTree Successfully installed PyIntervalTree-0.5 ➜ nextsv git:(master) sh install_pbhoney.sh #### installation of PBHoney #### ######################################################################## Please add the following environmental variables to your ~/.bashrc file: export SWEETPATH=/home/urbe/Tools/nextsv/aligners_and_callers/PBSuite_15.8.24 export PYTHONPATH=$PYTHONPATH:$SWEETPATH export PATH=$PATH:$SWEETPATH/bin ######################################################################## ➜ nextsv git:(master) ✗ sh install_sniffles.sh #### installation of Sniffles #### building Sniffles -- The C compiler identification is GNU 5.4.0 -- The CXX compiler identification is GNU 5.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- No build type specified. Using 'release' -- Try OpenMP C flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Try OpenMP CXX flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Found OpenMP: -fopenmp -- Looking for sys/types.h -- Looking for sys/types.h - found -- Looking for stdint.h -- Looking for stdint.h - found -- Looking for stddef.h -- Looking for stddef.h - found -- Check size of off64_t -- Check size of off64_t - done -- Looking for fseeko -- Looking for fseeko - found -- Looking for unistd.h -- Looking for unistd.h - found -- Configuring done -- Generating done -- Build files have been written to: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/build Scanning dependencies of target zlibstatic [ 1%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/adler32.o [ 2%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/compress.o [ 3%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/crc32.o [ 4%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/deflate.o [ 5%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/gzclose.o [ 6%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/gzlib.o [ 7%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/gzread.o [ 8%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/gzwrite.o [ 9%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/inflate.o [ 10%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/infback.o [ 11%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/inftrees.o [ 12%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/inffast.o [ 13%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/trees.o [ 14%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/uncompr.o [ 15%] Building C object lib/zlib-1.2.7/CMakeFiles/zlibstatic.dir/zutil.o [ 16%] Linking C static library libz.a [ 16%] Built target zlibstatic Scanning dependencies of target BamTools-static [ 17%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/BamAlignment.cpp.o [ 18%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/BamMultiReader.cpp.o [ 19%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/BamReader.cpp.o [ 20%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/BamWriter.cpp.o [ 21%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamHeader.cpp.o [ 22%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamProgram.cpp.o [ 23%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamProgramChain.cpp.o [ 24%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamReadGroup.cpp.o [ 25%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamReadGroupDictionary.cpp.o [ 26%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamSequence.cpp.o [ 27%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/SamSequenceDictionary.cpp.o [ 28%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/bam/BamHeader_p.cpp.o [ 29%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/bam/BamMultiReader_p.cpp.o [ 30%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/bam/BamRandomAccessController_p.cpp.o [ 31%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/bam/BamReader_p.cpp.o [ 32%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/bam/BamWriter_p.cpp.o [ 34%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/index/BamIndexFactory_p.cpp.o [ 35%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/index/BamStandardIndex_p.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/index/BamStandardIndex_p.cpp: In member function ‘void BamTools::Internal::BamStandardIndex::WriteLinearOffsets(const int&, BamTools::Internal::BaiLinearOffsetVector&)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/index/BamStandardIndex_p.cpp:958:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if ( numBytesWritten != (sizeof(offsetCount) + linearOffsets.size()*sizeof( ^ [ 36%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/index/BamToolsIndex_p.cpp.o [ 37%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BamDeviceFactory_p.cpp.o [ 38%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BamFile_p.cpp.o [ 39%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BamFtp_p.cpp.o [ 40%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BamHttp_p.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/io/BamHttp_p.cpp: In member function ‘bool BamTools::Internal::BamHttp::SendGetRequest(size_t)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/io/BamHttp_p.cpp:409:59: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if ( WriteToSocket(requestHeader.c_str(), headerSize) != headerSize ) { ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/io/BamHttp_p.cpp: In member function ‘bool BamTools::Internal::BamHttp::SendHeadRequest()’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/lib/bamtools-2.3.0/src/api/internal/io/BamHttp_p.cpp:501:59: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if ( WriteToSocket(requestHeader.c_str(), headerSize) != headerSize ) { ^ [ 41%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BamPipe_p.cpp.o [ 42%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/BgzfStream_p.cpp.o [ 43%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/ByteArray_p.cpp.o [ 44%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/HostAddress_p.cpp.o [ 45%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/HostInfo_p.cpp.o [ 46%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/HttpHeader_p.cpp.o [ 47%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/ILocalIODevice_p.cpp.o [ 48%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/RollingBuffer_p.cpp.o [ 49%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/TcpSocket_p.cpp.o [ 50%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/TcpSocketEngine_p.cpp.o [ 51%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/io/TcpSocketEngine_unix_p.cpp.o [ 52%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/sam/SamFormatParser_p.cpp.o [ 53%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/sam/SamFormatPrinter_p.cpp.o [ 54%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/sam/SamHeaderValidator_p.cpp.o [ 55%] Building CXX object lib/bamtools-2.3.0/src/api/CMakeFiles/BamTools-static.dir/internal/utils/BamException_p.cpp.o [ 56%] Linking CXX static library libbamtools.a [ 56%] Built target BamTools-static Scanning dependencies of target sniffles [ 57%] Building CXX object src/CMakeFiles/sniffles.dir/tree/Breakpoint_Tree.cpp.o [ 58%] Building CXX object src/CMakeFiles/sniffles.dir/Genotyper/Genotyper.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/Genotyper/Genotyper.cpp: In member function ‘void Genotyper::update_file(Breakpoint_Tree&, breakpoint_node*&)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/Genotyper/Genotyper.cpp:219:22: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result] system(move.c_str()); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/Genotyper/Genotyper.cpp: In member function ‘void Genotyper::update_SVs()’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/Genotyper/Genotyper.cpp:293:21: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result] system(del.c_str()); ^ [ 59%] Building CXX object src/CMakeFiles/sniffles.dir/Alignment.cpp.o [ 60%] Building CXX object src/CMakeFiles/sniffles.dir/BamParser.cpp.o [ 61%] Building CXX object src/CMakeFiles/sniffles.dir/Sniffles.cpp.o [ 62%] Building CXX object src/CMakeFiles/sniffles.dir/Ignore_Regions.cpp.o [ 63%] Building CXX object src/CMakeFiles/sniffles.dir/tree/Intervall_bed.cpp.o [ 64%] Building CXX object src/CMakeFiles/sniffles.dir/sub/Detect_Breakpoints.cpp.o [ 65%] Building CXX object src/CMakeFiles/sniffles.dir/sub/Breakpoint.cpp.o [ 67%] Building CXX object src/CMakeFiles/sniffles.dir/tree/IntervallTree.cpp.o [ 68%] Building CXX object src/CMakeFiles/sniffles.dir/tree/IntervallList.cpp.o [ 69%] Building CXX object src/CMakeFiles/sniffles.dir/realign/SWCPU.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/realign/SWCPU.cpp: In constructor ‘SWCPUCor::SWCPUCor(int)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/realign/SWCPU.cpp:30:71: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘long unsigned int’ [-Wformat=] fprintf(stderr, "Allocationg: %llu\n", maxLen * sizeof(MatrixElement)); ^ [ 70%] Building CXX object src/CMakeFiles/sniffles.dir/realign/Realign.cpp.o [ 71%] Building CXX object src/CMakeFiles/sniffles.dir/print/VCFPrinter.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/VCFPrinter.cpp: In member function ‘virtual void VCFPrinter::print_body(Breakpoint*&, BamTools::RefVector)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/VCFPrinter.cpp:154:41: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] fprintf(file, "%i", SV->get_length()); ^ [ 72%] Building CXX object src/CMakeFiles/sniffles.dir/print/BedpePrinter.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp: In member function ‘virtual void BedpePrinter::print_body(Breakpoint*&, BamTools::RefVector)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:35:89: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] file, "%i", IPrinter::calc_pos(SV->get_coordinates().start.max_pos, ref, chr)); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:42:88: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] (file, "%i", IPrinter::calc_pos(SV->get_coordinates().stop.max_pos, ref, chr)); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:70:41: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] fprintf(file, "%i", SV->get_length()); ^ [ 73%] Building CXX object src/CMakeFiles/sniffles.dir/print/IPrinter.cpp.o [ 74%] Building CXX object src/CMakeFiles/sniffles.dir/tree/BinTree.cpp.o [ 75%] Building CXX object src/CMakeFiles/sniffles.dir/print/NGMPrinter.cpp.o [ 76%] Building CXX object src/CMakeFiles/sniffles.dir/plane-sweep/PlaneSweep_slim.cpp.o [ 77%] Building CXX object src/CMakeFiles/sniffles.dir/cluster/Cluster_SVs.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/cluster/Cluster_SVs.cpp: In member function ‘void Cluster_SVS::update_SVs(std::vector<combine_str>&)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/cluster/Cluster_SVs.cpp:87:22: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result] system(move.c_str()); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/cluster/Cluster_SVs.cpp: In member function ‘void Cluster_SVS::update_SVs()’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/cluster/Cluster_SVs.cpp:159:21: warning: ignoring return value of ‘int system(const char*)’, declared with attribute warn_unused_result [-Wunused-result] system(del.c_str()); ^ [ 78%] Linking CXX executable ../../bin/sniffles-core-1.0.5/sniffles [ 78%] Built target sniffles Scanning dependencies of target sniffles-debug [ 79%] Building CXX object src/CMakeFiles/sniffles-debug.dir/tree/Breakpoint_Tree.cpp.o [ 80%] Building CXX object src/CMakeFiles/sniffles-debug.dir/Genotyper/Genotyper.cpp.o [ 81%] Building CXX object src/CMakeFiles/sniffles-debug.dir/Alignment.cpp.o [ 82%] Building CXX object src/CMakeFiles/sniffles-debug.dir/BamParser.cpp.o [ 83%] Building CXX object src/CMakeFiles/sniffles-debug.dir/Sniffles.cpp.o [ 84%] Building CXX object src/CMakeFiles/sniffles-debug.dir/Ignore_Regions.cpp.o [ 85%] Building CXX object src/CMakeFiles/sniffles-debug.dir/tree/Intervall_bed.cpp.o [ 86%] Building CXX object src/CMakeFiles/sniffles-debug.dir/tree/IntervallList.cpp.o [ 87%] Building CXX object src/CMakeFiles/sniffles-debug.dir/sub/Detect_Breakpoints.cpp.o [ 88%] Building CXX object src/CMakeFiles/sniffles-debug.dir/sub/Breakpoint.cpp.o [ 89%] Building CXX object src/CMakeFiles/sniffles-debug.dir/tree/IntervallTree.cpp.o [ 90%] Building CXX object src/CMakeFiles/sniffles-debug.dir/realign/SWCPU.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/realign/SWCPU.cpp: In constructor ‘SWCPUCor::SWCPUCor(int)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/realign/SWCPU.cpp:30:71: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘long unsigned int’ [-Wformat=] fprintf(stderr, "Allocationg: %llu\n", maxLen * sizeof(MatrixElement)); ^ [ 91%] Building CXX object src/CMakeFiles/sniffles-debug.dir/realign/Realign.cpp.o [ 92%] Building CXX object src/CMakeFiles/sniffles-debug.dir/print/VCFPrinter.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/VCFPrinter.cpp: In member function ‘virtual void VCFPrinter::print_body(Breakpoint*&, BamTools::RefVector)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/VCFPrinter.cpp:154:41: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] fprintf(file, "%i", SV->get_length()); ^ [ 93%] Building CXX object src/CMakeFiles/sniffles-debug.dir/print/BedpePrinter.cpp.o /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp: In member function ‘virtual void BedpePrinter::print_body(Breakpoint*&, BamTools::RefVector)’: /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:35:89: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] file, "%i", IPrinter::calc_pos(SV->get_coordinates().start.max_pos, ref, chr)); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:42:88: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] (file, "%i", IPrinter::calc_pos(SV->get_coordinates().stop.max_pos, ref, chr)); ^ /home/urbe/Tools/nextsv/aligners_and_callers/Sniffles-1.0.5/src/print/BedpePrinter.cpp:70:41: warning: format ‘%i’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=] fprintf(file, "%i", SV->get_length()); ^ [ 94%] Building CXX object src/CMakeFiles/sniffles-debug.dir/print/IPrinter.cpp.o [ 95%] Building CXX object src/CMakeFiles/sniffles-debug.dir/tree/BinTree.cpp.o [ 96%] Building CXX object src/CMakeFiles/sniffles-debug.dir/print/NGMPrinter.cpp.o [ 97%] Building CXX object src/CMakeFiles/sniffles-debug.dir/plane-sweep/PlaneSweep_slim.cpp.o [ 98%] Building CXX object src/CMakeFiles/sniffles-debug.dir/cluster/Cluster_SVs.cpp.o [100%] Linking CXX executable ../../bin/sniffles-core-1.0.5/sniffles-debug [100%] Built target sniffles-debug Installation of Sniffles finished ➜ nextsv git:(master) ✗]]> Jit https://bioinformaticsonline.com/snippets/view/37489/perl-script-to-find-palindromic-pattern Mon, 06 Aug 2018 09:28:25 -0500 https://bioinformaticsonline.com/snippets/view/37489/perl-script-to-find-palindromic-pattern #!/usr/bin/perl use strict; use warnings; use strict; my %invert; @invert{ qw[ A C G T ] } = qw[ T G C A ]; my $in = do{ local $/; <DATA> }; chomp $in; print $in; for my $p1 ( 1 .. length( $in ) -2 ) { next unless substr( $in, $p1, 1 ) eq $invert{ substr $in, $p1+1, 1 }; my $pals = 0; for my $p2 ( 1 .. $p1 -1 ) { last unless substr( $in, $p1-$p2, 1 ) eq $invert{ substr $in, $p1+$p2+1, 1 }; ++$pals; } if( $pals ) { my $val=substr( $in, $p1-$pals, ($pals+1)*2 ); my $st=$p1-$pals; my $sLen=($pals+1)*2; my $ed=($st+$sLen)-1; print "$st\t$ed\t$sLen\t$val\n"; printf "%s%s at %d\n", ' 'x($p1-$pals), substr( $in, $p1-$pals, ($pals+1)*2 ), $p1-$pals; } } __DATA__ AGAGGTCAGTCTGCATCGTATCGATCGTCGACGATCGATACGATGCAGACTGACGAGAG]]> Rahul Nayak https://bioinformaticsonline.com/snippets/view/37486/running-reapr Mon, 06 Aug 2018 08:24:15 -0500 https://bioinformaticsonline.com/snippets/view/37486/running-reapr set -e echo "Running reapr version 1.0.18 pipeline:" echo "/home/urbe/Tools/Reapr_1.0.18/reapr WadinetaGapFilled.gapfilled.iteration1.fa out.smaltmap.bam out.pipeline perfect" echo "[REAPR pipeline] Running facheck" echo "/home/urbe/Tools/Reapr_1.0.18/reapr facheck WadinetaGapFilled.gapfilled.iteration1.fa" echo "[REAPR pipeline] Running preprocess" echo "/home/urbe/Tools/Reapr_1.0.18/reapr preprocess WadinetaGapFilled.gapfilled.iteration1.fa out.smaltmap.bam out.pipeline" cd out.pipeline echo "[REAPR pipeline] Running stats" echo "/home/urbe/Tools/Reapr_1.0.18/reapr stats -p /home/urbe/Tools/Reapr_1.0.18/Wadineta_Afasta_gapfilled_genome/perfect.perfect_cov.gz ./ 01.stats" echo "[REAPR pipeline] Running fcdrate" /home/urbe/Tools/Reapr_1.0.18/reapr fcdrate ./ 01.stats 02.fcdrate fcdcutoff=`tail -n 1 02.fcdrate.info.txt | cut -f 1` echo "[REAPR pipeline] Running score" /home/urbe/Tools/Reapr_1.0.18/reapr score -P 5 00.assembly.fa.gaps.gz 00.in.bam 01.stats $fcdcutoff 03.score echo "[REAPR pipeline] Running break" /home/urbe/Tools/Reapr_1.0.18/reapr break 00.assembly.fa 03.score.errors.gff.gz 04.break echo "[REAPR pipeline] Running summary" /home/urbe/Tools/Reapr_1.0.18/reapr summary 00.assembly.fa 03.score 04.break 05.summary -------- set -e echo "Running reapr version 1.0.18 pipeline:" echo "/home/urbe/Tools/Reapr_1.0.18/reapr WadinetaGapFilled.gapfilled.iteration1.fa out.smaltmap.bam out.pipeline perfect" echo "[REAPR pipeline] Running facheck" /home/urbe/Tools/Reapr_1.0.18/reapr facheck WadinetaGapFilled.gapfilled.iteration1.fa echo "[REAPR pipeline] Running preprocess" /home/urbe/Tools/Reapr_1.0.18/reapr preprocess WadinetaGapFilled.gapfilled.iteration1.fa out.smaltmap.bam out.pipeline cd out.pipeline echo "[REAPR pipeline] Running stats" /home/urbe/Tools/Reapr_1.0.18/reapr stats -p /home/urbe/Tools/Reapr_1.0.18/Wadineta_Afasta_gapfilled_genome/perfect.perfect_cov.gz ./ 01.stats echo "[REAPR pipeline] Running fcdrate" /home/urbe/Tools/Reapr_1.0.18/reapr fcdrate ./ 01.stats 02.fcdrate fcdcutoff=`tail -n 1 02.fcdrate.info.txt | cut -f 1` echo "[REAPR pipeline] Running score" /home/urbe/Tools/Reapr_1.0.18/reapr score -P 5 00.assembly.fa.gaps.gz 00.in.bam 01.stats $fcdcutoff 03.score echo "[REAPR pipeline] Running break" /home/urbe/Tools/Reapr_1.0.18/reapr break 00.assembly.fa 03.score.errors.gff.gz 04.break echo "[REAPR pipeline] Running summary" /home/urbe/Tools/Reapr_1.0.18/reapr summary 00.assembly.fa 03.score 04.break 05.summary ----- #!/usr/bin/env bash set -e echo " test smaltmap " reapr smaltmap -u 5 -n 25 WadinetaGapFilled.gapfilled.iteration1.fa /home/urbe/DATA/OriginalReads/ANature/GC027568.151120_Adineta_Nature_and_Habrotrocha.160226.HiSeq2000.FCB.lane1.gcap_dev.R1.fastq /home/urbe/DATA/OriginalReads/ANature/GC027568.151120_Adineta_Nature_and_Habrotrocha.160226.HiSeq2000.FCB.lane1.gcap_dev.R2.fastq out.smaltmap.bam echo " smaltmap OK ______________________________________________________________________________ test perfectmap " reapr perfectmap WadinetaGapFilled.gapfilled.iteration1.fa /home/urbe/DATA/OriginalReads/ANature/GC027568.151120_Adineta_Nature_and_Habrotrocha.160226.HiSeq2000.FCB.lane1.gcap_dev.R1.fastq /home/urbe/DATA/OriginalReads/ANature/GC027568.151120_Adineta_Nature_and_Habrotrocha.160226.HiSeq2000.FCB.lane1.gcap_dev.R2.fastq 165 out.perfectmap echo " perfectmap OK ______________________________________________________________________________ test pipeline " reapr pipeline WadinetaGapFilled.gapfilled.iteration1.fa out.smaltmap.bam out.pipeline perfect echo " pipeline OK ______________________________________________________________________________ All looked OK so cleaning files... " #rm -fr out.* echo " All done! "]]> Jit https://bioinformaticsonline.com/snippets/view/37451/installing-porechop-on-ubuntu Mon, 30 Jul 2018 10:05:45 -0500 https://bioinformaticsonline.com/snippets/view/37451/installing-porechop-on-ubuntu ➜ Tools git:(master) ✗ git clone https://github.com/rrwick/Porechop.git Cloning into 'Porechop'... remote: Counting objects: 1579, done. remote: Total 1579 (delta 0), reused 0 (delta 0), pack-reused 1579 Receiving objects: 100% (1579/1579), 3.38 MiB | 1.62 MiB/s, done. Resolving deltas: 100% (869/869), done. Checking connectivity... done. ➜ Tools git:(master) ✗ cd Porechop ➜ Porechop git:(master) python3 setup.py install running install running build running build_py creating build creating build/lib creating build/lib/porechop copying porechop/version.py -> build/lib/porechop copying porechop/nanopore_read.py -> build/lib/porechop copying porechop/cpp_function_wrappers.py -> build/lib/porechop copying porechop/porechop.py -> build/lib/porechop copying porechop/misc.py -> build/lib/porechop copying porechop/adapters.py -> build/lib/porechop copying porechop/__init__.py -> build/lib/porechop Cleaning previous compilation: make clean rm -f porechop/src/adapter_align.o porechop/src/alignment.o Compiling Porechop: make -j 8 g++ -std=c++14 -Iporechop/include -fPIC -O3 -D NDEBUG -Wall -Wextra -pedantic -mtune=native -c -o porechop/src/adapter_align.o porechop/src/adapter_align.cpp g++ -std=c++14 -Iporechop/include -fPIC -O3 -D NDEBUG -Wall -Wextra -pedantic -mtune=native -c -o porechop/src/alignment.o porechop/src/alignment.cpp g++ -std=c++14 -Iporechop/include -fPIC -O3 -D NDEBUG -Wall -Wextra -pedantic -mtune=native -shared -Wl,-soname,porechop/cpp_functions.so -o porechop/cpp_functions.so porechop/src/adapter_align.o porechop/src/alignment.o running install_lib creating /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/version.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/nanopore_read.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/cpp_function_wrappers.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/porechop.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/misc.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/adapters.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop copying build/lib/porechop/__init__.py -> /home/urbe/anaconda3/lib/python3.6/site-packages/porechop byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/version.py to version.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/nanopore_read.py to nanopore_read.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/cpp_function_wrappers.py to cpp_function_wrappers.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/porechop.py to porechop.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/misc.py to misc.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/adapters.py to adapters.cpython-36.pyc byte-compiling /home/urbe/anaconda3/lib/python3.6/site-packages/porechop/__init__.py to __init__.cpython-36.pyc running install_egg_info running egg_info creating porechop.egg-info writing porechop.egg-info/PKG-INFO writing dependency_links to porechop.egg-info/dependency_links.txt writing entry points to porechop.egg-info/entry_points.txt writing top-level names to porechop.egg-info/top_level.txt writing manifest file 'porechop.egg-info/SOURCES.txt' reading manifest file 'porechop.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'porechop.egg-info/SOURCES.txt' Copying porechop.egg-info to /home/urbe/anaconda3/lib/python3.6/site-packages/porechop-0.2.3-py3.6.egg-info running install_scripts Installing porechop script to /home/urbe/anaconda3/bin ➜ Porechop git:(master) porechop -h usage: porechop -i INPUT [-o OUTPUT] [--format {auto,fasta,fastq,fasta.gz,fastq.gz}] [-v VERBOSITY] [-t THREADS] [-b BARCODE_DIR] [--barcode_threshold BARCODE_THRESHOLD] [--barcode_diff BARCODE_DIFF] [--require_two_barcodes] [--untrimmed] [--discard_unassigned] [--adapter_threshold ADAPTER_THRESHOLD] [--check_reads CHECK_READS] [--scoring_scheme SCORING_SCHEME] [--end_size END_SIZE] [--min_trim_size MIN_TRIM_SIZE] [--extra_end_trim EXTRA_END_TRIM] [--end_threshold END_THRESHOLD] [--no_split] [--discard_middle] [--middle_threshold MIDDLE_THRESHOLD] [--extra_middle_trim_good_side EXTRA_MIDDLE_TRIM_GOOD_SIDE] [--extra_middle_trim_bad_side EXTRA_MIDDLE_TRIM_BAD_SIDE] [--min_split_read_size MIN_SPLIT_READ_SIZE] [-h] [--version] Porechop: a tool for finding adapters in Oxford Nanopore reads, trimming them from the ends and splitting reads with internal adapters Main options: -i INPUT, --input INPUT FASTA/FASTQ of input reads or a directory which will be recursively searched for FASTQ files (required) -o OUTPUT, --output OUTPUT Filename for FASTA or FASTQ of trimmed reads (if not set, trimmed reads will be printed to stdout) --format {auto,fasta,fastq,fasta.gz,fastq.gz} Output format for the reads - if auto, the format will be chosen based on the output filename or the input read format (default: auto) -v VERBOSITY, --verbosity VERBOSITY Level of progress information: 0 = none, 1 = some, 2 = lots, 3 = full - output will go to stdout if reads are saved to a file and stderr if reads are printed to stdout (default: 1) -t THREADS, --threads THREADS Number of threads to use for adapter alignment (default: 16) Barcode binning settings: Control the binning of reads based on barcodes (i.e. barcode demultiplexing) -b BARCODE_DIR, --barcode_dir BARCODE_DIR Reads will be binned based on their barcode and saved to separate files in this directory (incompatible with --output) --barcode_threshold BARCODE_THRESHOLD A read must have at least this percent identity to a barcode to be binned (default: 75.0) --barcode_diff BARCODE_DIFF If the difference between a read's best barcode identity and its second-best barcode identity is less than this value, it will not be put in a barcode bin (to exclude cases which are too close to call) (default: 5.0) --require_two_barcodes Reads will only be put in barcode bins if they have a strong match for the barcode on both their start and end (default: a read can be binned with a match at its start or end) --untrimmed Bin reads but do not trim them (default: trim the reads) --discard_unassigned Discard unassigned reads (instead of creating a "none" bin) (default: False) Adapter search settings: Control how the program determines which adapter sets are present --adapter_threshold ADAPTER_THRESHOLD An adapter set has to have at least this percent identity to be labelled as present and trimmed off (0 to 100) (default: 90.0) --check_reads CHECK_READS This many reads will be aligned to all possible adapters to determine which adapter sets are present (default: 10000) --scoring_scheme SCORING_SCHEME Comma-delimited string of alignment scores: match, mismatch, gap open, gap extend (default: 3,-6,-5,-2) End adapter settings: Control the trimming of adapters from read ends --end_size END_SIZE The number of base pairs at each end of the read which will be searched for adapter sequences (default: 150) --min_trim_size MIN_TRIM_SIZE Adapter alignments smaller than this will be ignored (default: 4) --extra_end_trim EXTRA_END_TRIM This many additional bases will be removed next to adapters found at the ends of reads (default: 2) --end_threshold END_THRESHOLD Adapters at the ends of reads must have at least this percent identity to be removed (0 to 100) (default: 75.0) Middle adapter settings: Control the splitting of read from middle adapters --no_split Skip splitting reads based on middle adapters (default: split reads when an adapter is found in the middle) --discard_middle Reads with middle adapters will be discarded (default: reads with middle adapters are split) (required for reads to be used with Nanopolish, this option is on by default when outputting reads into barcode bins) --middle_threshold MIDDLE_THRESHOLD Adapters in the middle of reads must have at least this percent identity to be found (0 to 100) (default: 85.0) --extra_middle_trim_good_side EXTRA_MIDDLE_TRIM_GOOD_SIDE This many additional bases will be removed next to middle adapters on their "good" side (default: 10) --extra_middle_trim_bad_side EXTRA_MIDDLE_TRIM_BAD_SIDE This many additional bases will be removed next to middle adapters on their "bad" side (default: 100) --min_split_read_size MIN_SPLIT_READ_SIZE Post-split read pieces smaller than this many base pairs will not be outputted (default: 1000) Help: -h, --help Show this help message and exit --version Show program's version number and exit]]> Jit https://bioinformaticsonline.com/snippets/view/37293/running-transrate Fri, 13 Jul 2018 07:55:46 -0500 https://bioinformaticsonline.com/snippets/view/37293/running-transrate ➜ BorisTrinityAssembly /home/urbe/Tools/transrate-1.0.3-linux-x86_64/transrate _ _ | |_ _ __ __ _ _ __ ___ _ __ __ _ | |_ ___ ░▓▓▓^▓▓▓░ | __|| '__|/ _` || '_ \ / __|| '__|/ _` || __|/ _ \ ░▓▓▓^▓▓▓░ ░▓▓▓^▓▓▓░ | |_ | | | (_| || | | |\__ \| | | (_| || |_| __/ ░▓▓▓^▓▓▓░ ░▓▓▓^▓▓▓░ \__||_| \__,_||_| |_||___/|_| \__,_| \__|\___| ░▓▓▓^▓▓▓░ Transrate v1.0.3 by Richard Smith-Unna, Chris Boursnell, Rob Patro, Julian Hibberd, and Steve Kelly DESCRIPTION: Analyse a de-novo transcriptome assembly using three kinds of metrics: 1. sequence based (if --assembly is given) 2. read mapping based (if --left and --right are given) 3. reference based (if --reference is given) Documentation at http://hibberdlab.com/transrate USAGE: transrate <options> OPTIONS: --assembly=<s> Assembly file(s) in FASTA format, comma-separated --left=<s> Left reads file(s) in FASTQ format, comma-separated --right=<s> Right reads file(s) in FASTQ format, comma-separated --reference=<s> Reference proteome or transcriptome file in FASTA format --threads=<i> Number of threads to use (default: 8) --merge-assemblies=<s> Merge best contigs from multiple assemblies into file --output=<s> Directory where results are output (will be created) (default: transrate_results) --loglevel=<s> Log level. One of [error, info, warn, debug] (default: info) --install-deps=<s> Install any missing dependencies. One of [ref] --examples Show some example commands with explanations ➜ BorisTrinityAssembly /home/urbe/Tools/transrate-1.0.3-linux-x86_64/transrate --assembly=Trinity.fasta --output=transrateSTAT [ INFO] 2018-07-13 14:54:03 : Loading assembly: /media/urbe/MyPassport/BorisTrinityAssembly/Trinity.fasta [ INFO] 2018-07-13 14:54:21 : Analysing assembly: /media/urbe/MyPassport/BorisTrinityAssembly/Trinity.fasta [ INFO] 2018-07-13 14:54:21 : Results will be saved in /media/urbe/MyPassport/BorisTrinityAssembly/transrateSTAT/Trinity [ INFO] 2018-07-13 14:54:21 : Calculating contig metrics... [ INFO] 2018-07-13 14:54:34 : Contig metrics: [ INFO] 2018-07-13 14:54:34 : ----------------------------------- [ INFO] 2018-07-13 14:54:34 : n seqs 98389 [ INFO] 2018-07-13 14:54:34 : smallest 201 [ INFO] 2018-07-13 14:54:34 : largest 22136 [ INFO] 2018-07-13 14:54:34 : n bases 147938374 [ INFO] 2018-07-13 14:54:34 : mean len 1503.61 [ INFO] 2018-07-13 14:54:34 : n under 200 0 [ INFO] 2018-07-13 14:54:34 : n over 1k 53685 [ INFO] 2018-07-13 14:54:34 : n over 10k 88 [ INFO] 2018-07-13 14:54:34 : n with orf 61028 [ INFO] 2018-07-13 14:54:34 : mean orf percent 61.07 [ INFO] 2018-07-13 14:54:34 : n90 748 [ INFO] 2018-07-13 14:54:34 : n70 1578 [ INFO] 2018-07-13 14:54:34 : n50 2286 [ INFO] 2018-07-13 14:54:34 : n30 3182 [ INFO] 2018-07-13 14:54:34 : n10 4912 [ INFO] 2018-07-13 14:54:34 : gc 0.33 [ INFO] 2018-07-13 14:54:34 : bases n 0 [ INFO] 2018-07-13 14:54:34 : proportion n 0.0 [ INFO] 2018-07-13 14:54:34 : Contig metrics done in 13 seconds [ INFO] 2018-07-13 14:54:34 : No reads provided, skipping read diagnostics [ INFO] 2018-07-13 14:54:34 : No reference provided, skipping comparative diagnostics [ INFO] 2018-07-13 14:54:34 : Writing contig metrics for each contig to /media/urbe/MyPassport/BorisTrinityAssembly/transrateSTAT/Trinity/contigs.csv [ INFO] 2018-07-13 14:54:40 : Writing analysis results to assemblies.csv]]> Jit https://bioinformaticsonline.com/snippets/view/37283/long-reads-mapper-bash-script Thu, 12 Jul 2018 10:15:44 -0500 https://bioinformaticsonline.com/snippets/view/37283/long-reads-mapper-bash-script #!/bin/bash #only for LONG READS mapping #USAGE: runMapper.sh minimap2 ref.fa reads.fa/fq 48 ont echo -e "This is a bash scrip to map your long reads and make it visualization ready !" #Location of the tools samtools=/home/urbe/anaconda3/bin/samtools graphMapLoc=/home/urbe/Tools/graphmap/bin/Linux-x64/graphmap minimap2Loc=/home/urbe/Tools/minimap2-2.3_x64-linux/minimap2 bwaMemLoc=/home/urbe/anaconda3/bin/bwa ngmlrLoc=/home/urbe/Tools/ngmlr/bin/ngmlr-0.2.3/ngmlr lamsaLoc=/home/urbe/Tools/LAMSA/lamsa #Parameters accepted toolName=$1 refFasta=$2 longReads=$3 thread=$4 readsType=$5 if [ $# -lt 5 ]; then echo "No or less arguments provided" echo "#USAGE: runMapper.sh minimap2 ref.fa reads.fa/fq 48 ont" exit 1 fi fileName=$(basename "$refFasta"); #fileName=$(basename "$refFasta" .fq); echo "Name of the file used for mapping $fileName, present at $refFasta" if [ $toolName == "bwa" ]; then echo "Mapping with $toolName" $bwaMemLoc index $refFasta $bwaMemLoc mem $refFasta $longReads -t $thread > $fileName.out.sam elif [ $toolName == "lamsa" ]; then echo "Mapping with $toolName" $lamsaLoc index $refFasta if [ $readsType == "ont" ]; then $lamsaLoc aln -t $thread -T ont2d $refFasta $longReads > $fileName.out.sam elif [ $readsType == "pacbio" ]; then $lamsaLoc aln -t $thread -T pacbio $refFasta $longReads > $fileName.out.sam else echo "LAMSA:Please specify reads type: ont, pacbio" fi elif [ $toolName == "minimap2" ]; then echo "Mapping with $toolName" if [ $readsType == "ont" ]; then $minimap2Loc -ax map-ont $refFasta $longReads -t $thread > $fileName.out.sam elif [ $readsType == "pacbio" ]; then $minimap2Loc -ax map-pb $refFasta $longReads -t $thread > $fileName.out.sam else echo "MINIMAP2:Please specify reads type: ont, pacbio" fi elif [ $toolName == "ngmlr" ]; then echo "Mapping with $toolName" if [ $readsType == "ont" ]; then $ngmlrLoc -t $thread -r $refFasta -q $longReads -o $fileName.out.sam -x $readsType elif [ $readsType == "pacbio" ]; then $ngmlrLoc -t $thread -r $refFasta -q $longReads -o $fileName.out.sam -x $readsType else echo "NGMLR:Please specify reads type: ont, pacbio" fi elif [ $toolName == "graphmap" ]; then echo "Mapping with $toolName" $graphMapLoc align -r $refFasta -d $longReads -t $thread -o $fileName.out.sam else echo "Unknown mapper name and parameter !" fi echo "Getting files ready for visualization !" $samtools view -Sb $fileName.out.sam | $samtools sort -m 4G -@$thread -o $fileName.sorted.bam - && $samtools index -@$thread $fileName.sorted.bam echo "All Done :)"]]> Jit https://bioinformaticsonline.com/snippets/view/37277/perl-script-to-merge-lastz-overlaps Wed, 11 Jul 2018 04:13:51 -0500 https://bioinformaticsonline.com/snippets/view/37277/perl-script-to-merge-lastz-overlaps #!/usr/bin/perl use strict; use warnings; use 5.010; # Filter out the exact/direct overlaps from tab seperated alignment file. (lastz format=general- ready) # Do not inclide header in lastz outfile # USAGE: perl filterOverlaps.pl infile > outfile open my $fh, '<', $ARGV[0]; my @terms; while (<$fh>) { chomp; push @terms, [split /\t/]; } my $biggest = 0; my $id = ''; for my $term (sort sorter @terms) { $biggest = 0 if $id ne $term->[0]; if ($term->[2] > $biggest) { say join "\t", @$term; $biggest = $term->[2]; } $id = $term->[0]; } sub sorter { $a->[0] cmp $b->[0] || $a->[1] <=> $b->[1] || $b->[2] <=> $a->[2] }]]> Jit https://bioinformaticsonline.com/snippets/view/37268/fastq-dump-for-sra-download Tue, 10 Jul 2018 05:02:20 -0500 https://bioinformaticsonline.com/snippets/view/37268/fastq-dump-for-sra-download ➜ bin git:(master) ✗ ./fastq-dump --help Usage: ./fastq-dump [options] <path> [<path>...] ./fastq-dump [options] <accession> INPUT -A|--accession <accession> Replaces accession derived from <path> in filename(s) and deflines (only for single table dump) --table <table-name> Table name within cSRA object, default is "SEQUENCE" PROCESSING Read Splitting Sequence data may be used in raw form or split into individual reads --split-spot Split spots into individual reads Full Spot Filters Applied to the full spot independently of --split-spot -N|--minSpotId <rowid> Minimum spot id -X|--maxSpotId <rowid> Maximum spot id --spot-groups <[list]> Filter by SPOT_GROUP (member): name[,...] -W|--clip Remove adapter sequences from reads Common Filters Applied to spots when --split-spot is not set, otherwise - to individual reads -M|--minReadLen <len> Filter by sequence length >= <len> -R|--read-filter <[filter]> Split into files by READ_FILTER value optionally filter by value: pass|reject|criteria|redacted -E|--qual-filter Filter used in early 1000 Genomes data: no sequences starting or ending with >= 10N --qual-filter-1 Filter used in current 1000 Genomes data Filters based on alignments Filters are active when alignment data are present --aligned Dump only aligned sequences --unaligned Dump only unaligned sequences --aligned-region <name[:from-to]> Filter by position on genome. Name can either be accession.version (ex: NC_000001.10) or file specific name (ex: "chr1" or "1"). "from" and "to" are 1-based coordinates --matepair-distance <from-to|unknown> Filter by distance between matepairs. Use "unknown" to find matepairs split between the references. Use from-to to limit matepair distance on the same reference Filters for individual reads Applied only with --split-spot set --skip-technical Dump only biological reads OUTPUT -O|--outdir <path> Output directory, default is working directory '.' ) -Z|--stdout Output to stdout, all split data become joined into single stream --gzip Compress output using gzip --bzip2 Compress output using bzip2 Multiple File Options Setting these options will produce more than 1 file, each of which will be suffixed according to splitting criteria. --split-files Dump each read into separate file.Files will receive suffix corresponding to read number --split-3 Legacy 3-file splitting for mate-pairs: First biological reads satisfying dumping conditions are placed in files *_1.fastq and *_2.fastq If only one biological read is present it is placed in *.fastq Biological reads and above are ignored. -G|--spot-group Split into files by SPOT_GROUP (member name) -R|--read-filter <[filter]> Split into files by READ_FILTER value optionally filter by value: pass|reject|criteria|redacted -T|--group-in-dirs Split into subdirectories instead of files -K|--keep-empty-files Do not delete empty files FORMATTING Sequence -C|--dumpcs <[cskey]> Formats sequence using color space (default for SOLiD),"cskey" may be specified for translation -B|--dumpbase Formats sequence using base space (default for other than SOLiD). Quality -Q|--offset <integer> Offset to use for quality conversion, default is 33 --fasta <[line width]> FASTA only, no qualities, optional line wrap width (set to zero for no wrapping) --suppress-qual-for-cskey suppress quality-value for cskey Defline -F|--origfmt Defline contains only original sequence name -I|--readids Append read id after spot id as 'accession.spot.readid' on defline --helicos Helicos style defline --defline-seq <fmt> Defline format specification for sequence. --defline-qual <fmt> Defline format specification for quality. <fmt> is string of characters and/or variables. The variables can be one of: $ac - accession, $si spot id, $sn spot name, $sg spot group (barcode), $sl spot length in bases, $ri read number, $rn read name, $rl read length in bases. '[]' could be used for an optional output: if all vars in [] yield empty values whole group is not printed. Empty value is empty string or for numeric variables. Ex: @$sn[_$rn]/$ri '_$rn' is omitted if name is empty OTHER: --disable-multithreading disable multithreading -h|--help Output brief explanation of program usage -V|--version Display the version of the program -L|--log-level <level> Logging level as number or enum string One of (fatal|sys|int|err|warn|info) or (0-5) Current/default is warn -v|--verbose Increase the verbosity level of the program Use multiple times for more verbosity --ncbi_error_report Control program execution environment report generation (if implemented). One of (never|error|always). Default is error --legacy-report use legacy style 'Written spots' for tool ./fastq-dump : 2.9.1 ( 2.9.1-1 )]]> Jit