Module Expr Str Import alias ImportFrom alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias alias FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call NameLoad NameLoad Assign NameStore Str Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr Str keyword NameLoad keyword Str keyword Str keyword Num keyword Str Assign NameStore Call NameLoad NameLoad keyword NameLoad Return NameLoad decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore BinOpAdd BinOpAdd Str Str Str Assign NameStore Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore BinOpAdd BinOpAdd AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword NameLoad Call AttributeLoad Str attr keyword NameLoad If CompareIsNot NameLoad NameLoad body AugAssignAdd NameStore Call AttributeLoad Str attr keyword NameLoad If CompareIsNot NameLoad NameLoad body AugAssignAdd NameStore Call AttributeLoad Str attr keyword NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad keyword Call AttributeLoad Str attr keyword NameLoad keyword NameLoad keyword NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Str If UnaryOpNot Call NameLoad NameLoad body Assign NameStore BinOpAdd BinOpAdd Call AttributeLoad Str attr keyword NameLoad Call AttributeLoad Str attr keyword NameLoad Str orelse If UnaryOpNot Call NameLoad NameLoad body Assign NameStore BinOpAdd BinOpAdd Str Call AttributeLoad Str attr keyword NameLoad Str orelse If UnaryOpNot Call NameLoad NameLoad body Assign NameStore BinOpAdd BinOpAdd Str Call AttributeLoad Str attr keyword NameLoad Str If CompareNotEq Call NameLoad NameLoad Num body Raise Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad With Call NameLoad NameLoad Str NameStore body Expr Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword NameLoad Expr Call AttributeLoad NameLoad attr Return TupleLoad NameLoad NameLoad NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Expr Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword AttributeLoad NameLoad attr keyword NameLoad decorator_list decorator_list
N/A N/A 
Overview:
    pbtranscript cluster contains two main components:
    * (1) ICE (iterative clustering and error correction) to predict
      unpolished consensus isoforms.
    * (2) Polish, to use nfl reads and quiver to polish those predicted
      unpolished isoforms. Polish contains three steps:
      + (2.1) IceAllPartials (ice_partial.py all)
              Align and assign nfl reads to unpolished isoforms, and
              save results to a pickle file.
      + (2.2) IceQuiver (ice_quiver.py all)
              Call quiver to polish each isoform based on alignments
              created by mapping its associated fl and nfl reads to
              this isoform.
      + (2.3) IceQuiverPostprocess (ice_quiver.py postprocess)
              Collect and post process quiver results, and classify
              HQ/LQ isoforms.

    In order to handle subtasks by SMRTPipe instead of pbtranscript
    itself, we will refactor the polish phase including
    (2.1) (2.2) and (2.3).

    (2.1) IceAllPartials (ice_partial.py all) will be refactored to
      + (2.1.1) ice_partial.py split
                Split nfl reads into N chunks (N<=100).
      + (2.1.2) ice_partial.py i
                For each chunk i, align and assign its reads to unpolished
                isoforms and create a pickle file.
      + (2.1.3) ice_partial.py merge
                Merge pickles for all splitted chunks together to a
                big pickle.

    *** Here we are focusing on (2.1.2) ice_partial.py i ***

Description:
    (2.1.2) ice_partial.py i

    Assumption:
     * Phase (1) ICE is done and unpolished isoforms created.
     * Step (2.1.1) is done, and nfl reads have been splitted into N
       chunks.
     * We are processing the i-th chunk of nfl reads.

    Process:
        Given root_dir, and i, aligns reads in the i-th chunk of nfl reads
        to unpolished consensus isoforms and creates an output pickle file.

        The i-th chunk is:
            root_dir/output/map_noFL/input.split_{0:02d}.fa
        The unpolished isoforms are saved in:
            root_dir/output/final.consensus.fa
        If available, the suffix array for unpolished isoforms is at:
            root_dir/output/final.consensus.fa.sa
        The output pickle file for the i-th chunk is:
            root_dir/output/map_noFL/input.split_{0:02d}.fa.partial_uc.pickle
        The done file to indicate that this job is successfully completed:
            root_dir/output/map_noFL/input.split_{0:02d}.fa.partial_uc.pickle.DONE
        The number of chunks, N, is not nececssary for this script.

        Log this commnd to a script file for debugging:
            root_dir/scripts/input.splilt_{0:02d}.fa.partial_uc.sh

    Input:
        Positional:
            root_dir, an output directory for running pbtranscript cluster.
            i, the i-th chunk, 0 <= i < N.
        Optional:
            ccs_fofn, A FOFN of ccs.h5 files (e.g., reads_of_insert.fofn),
            which contain quality values of reads of insert.
            If not given, assume there is no QV information available.
        blasr_nproc: Number of blasr nproc.

    Output:
        Align reads in the i-th chunk to unpolished consensus isoforms
        using BLASR, assign reads to isoforms and then save the following
        to a pickle:
            {isoform_id:[read_ids], nohit: set(no_hit_read_ids)}
        touch a done file when this job is finished.

    Hierarchy:
        pbtranscript = iceiterative

        pbtranscript --quiver = iceiterative +                                 ice_polish.py

        ice_polish.py =  ice_make_fasta_fofn.py +                          ice_partial.py all +                          ice_quiver.py all

        ice_partial.py all = ice_partial.py split +                              ice_partial.py i +                              ice_partial.py merge

        (ice_partial.py one --> only apply ice_partial on a given input fasta)

        ice_quiver.py all = ice_quiver.py i +                             ice_quiver.py merge +                             ice_quiver.py postprocess

    Example:
        ice_partial.py i root_dir {i} --ccs_fofn=ccs_fofn                          --blasr_nproc=blasr_nproc

 N/A logging pbtools.pbtranscript.__init__ get_version pbtools.pbtranscript.PBTranscriptOptions add_fofn_arguments add_cluster_root_dir_as_positional_argument pbtools.pbtranscript.Utils nfs_exists pbtools.pbtranscript.ice.IceFiles IceFiles pbtools.pbtranscript.ice.IcePartial build_uc_from_partial build_uc_from_partial_daligner add_ice_partial_i_arguments N/A N/A parser N/A N/A N/A Add IcePartialI arguments. N/A parser N/A add_cluster_root_dir_as_positional_argument parser N/A helpstr To process the i-th chunk of non-full-length reads. N/A N/A N/A parser add_argument i help helpstr type int N/A N/A N/A parser add_argument --blasr_nproc type int dest blasr_nproc action store default 12 help Number of cores for each BLASR job. N/A parser N/A add_fofn_arguments parser ccs_fofn True N/A parser N/A IcePartialI N/A object N/A N/A IcePartialI object. N/A desc N/A N/A Process the i-th chunk of non-full-length reads, align  and assign these reads to unpolished consensus isoforms  and create a pickle file. N/A prog ice_partial.py i  __init__ N/A N/A self root_dir i ccs_fofn blasr_nproc N/A N/A N/A 
        root_dir --- root directory for saving intermediate files running
        pbtranscript cluster.
        i --- the i-th chunk of nfl reads to assign to isoforms
        ccs_fofn --- FOFN of ccs.h5
        blasr_nproc --- blasr nproc
         N/A N/A self root_dir root_dir N/A N/A self i N/A int i N/A N/A self ccs_fofn ccs_fofn N/A N/A self blasr_nproc N/A int blasr_nproc N/A getVersion N/A N/A self N/A N/A N/A Return version string. N/A N/A get_version N/A cmd_str N/A N/A self N/A N/A N/A Return a cmd string of this object. N/A N/A N/A self _cmd_str root_dir N/A self root_dir i N/A self i ccs_fofn N/A self ccs_fofn blasr_nproc N/A self blasr_nproc N/A _cmd_str N/A N/A self root_dir i ccs_fofn blasr_nproc N/A N/A N/A Return a cmd string given parameters. N/A cmd N/A N/A N/A self prog N/A N/A {d}  format d root_dir N/A N/A {i}  format i i N/A N/A ccs_fofn None N/A N/A cmd N/A N/A --ccs_fofn={ccs_fofn}  format ccs_fofn ccs_fofn N/A N/A blasr_nproc None N/A N/A cmd N/A N/A --blasr_nproc={n}  format n blasr_nproc N/A cmd N/A validate_inputs N/A N/A self N/A N/A N/A Check inputs, and write ice_partial.py i command to script file,
        return
        (input_fasta, ref_fasta, dazz_db, out_pickle, done_file)
        for the i-th chunk of nfl reads.
         N/A N/A N/A self _validate_inputs root_dir N/A self root_dir i N/A self i ccs_fofn N/A self ccs_fofn blasr_nproc N/A self blasr_nproc N/A _validate_inputs N/A N/A self root_dir i ccs_fofn blasr_nproc N/A N/A N/A 
        Check inputs, write ice_partial.py i command to script_file
        and return (input_fasta, ref_fasta, sa_file, out_pickle, done_file)
        for the i-th chunk of nfl reads.
         N/A icef N/A IceFiles prog_name N/A N/A ice_partial_{i} format i i root_dir root_dir no_log_f False N/A ref_fasta N/A icef final_consensus_fa N/A ref_dazz N/A icef final_dazz_db N/A input_fasta N/A N/A icef nfl_fa_i i N/A out_pickle N/A N/A icef nfl_pickle_i i N/A done_file N/A N/A icef nfl_done_i i N/A script_file N/A N/A icef nfl_script_i i N/A errMsg N/A N/A N/A nfs_exists input_fasta N/A N/A errMsg N/A N/A N/A N/A The {i}-th splitted non-full-length reads  format i i N/A N/A fasta file {f} does not exist.  format f input_fasta Please run ice_partial_split.py first. N/A N/A N/A N/A nfs_exists ref_fasta N/A N/A errMsg N/A N/A The unpolished consensus isoforms fasta file  N/A N/A {f} does not exist.  format f ref_fasta Please make sure ICE is successfully done. N/A N/A N/A N/A nfs_exists ref_dazz N/A N/A errMsg N/A N/A The dazz db  N/A N/A {f} does not exist.  format f ref_dazz Please make sure it is already built. N/A N/A N/A len errMsg 0 N/A N/A N/A ValueError errMsg N/A cmd N/A N/A self _cmd_str root_dir root_dir i i ccs_fofn ccs_fofn blasr_nproc blasr_nproc N/A N/A open script_file w writer N/A N/A N/A N/A writer write N/A cmd N/A N/A N/A icef add_log N/A N/A Writing CMD to: {script_file} format script_file script_file N/A N/A N/A icef close_log N/A N/A input_fasta ref_fasta ref_dazz out_pickle done_file N/A run N/A N/A self N/A N/A N/A Run IcePartialI N/A N/A N/A logging debug N/A N/A root_dir: {d}. format d N/A self root_dir N/A N/A N/A logging debug N/A N/A i: {f}. format f N/A self i N/A N/A N/A logging debug N/A N/A ccs_fofn: {ccs_fofn}. format ccs_fofn N/A self ccs_fofn N/A N/A input_fasta ref_fasta dazz_db out_pickle done_file N/A N/A self validate_inputs N/A N/A N/A logging info Calling build_uc_from_partial using DALIGNER. N/A N/A build_uc_from_partial_daligner input_fasta input_fasta ref_fasta ref_fasta out_pickle out_pickle ccs_fofn N/A self ccs_fofn done_filename done_file use_finer_qv False cpus N/A self blasr_nproc no_qv_or_aln_checking True N/A N/A