opened
phu5ion
username_0
Hi Brad,

I'm having problems reading paired end fastq.gz files into bcbio. I ensured that the config file follows the same format as specified in the documentation, but it looks like multiple files in a list are not supported?

Trace:
[Wed Sep 07 14:56:07 ~/storage3/hodl/Projects/180/split_reads/structural_variants/bcbio/work ] $bcbio_nextgen.py ../config/config.yaml -n 8
[2016-09-07T07:22Z] System YAML configuration: /home/hodl/local/share/bcbio/galaxy/bcbio_system.yaml
[2016-09-07T07:22Z] Resource requests: bwa, sambamba, samtools; memory: 2.00, 2.00; cores: 16, 16, 16
[2016-09-07T07:22Z] Configuring 1 jobs to run, using 8 cores each with 16.1g of memory reserved for each job
[2016-09-07T07:22Z] Timing: organize samples
[2016-09-07T07:22Z] multiprocessing: organize_samples
[2016-09-07T07:22Z] Using input YAML configuration: /media/storage3/hodl/Projects/180/split_reads/structural_variants/bcbio/config/config.yaml
Traceback (most recent call last):
  File "/home/hodl/local/bin/bcbio_nextgen.py", line 4, in <module>
    __import__('pkg_resources').run_script('bcbio-nextgen==0.9.9', 'bcbio_nextgen.py')
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 726, in run_script

  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 1484, in run_script

  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.9-py2.7.egg-info/scripts/bcbio_nextgen.py", line 226, in <module>
    main(**kwargs)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.9-py2.7.egg-info/scripts/bcbio_nextgen.py", line 43, in main
    run_main(**kwargs)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 43, in run_main
    fc_dir, run_info_yaml)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 87, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 120, in variant2pipeline
    [x[0]["description"] for x in samples]]])
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 800, in __call__
    while self.dispatch_one_batch(iterator):
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 658, in dispatch_one_batch
    self._dispatch(tasks)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 566, in _dispatch
    job = ImmediateComputeBatch(batch)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 180, in __init__
    self.results = batch()
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper
    return apply(f, *args, **kwargs)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 300, in organize_samples
    return run_info.organize(*args)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 47, in organize
    run_details = _run_info_from_yaml(dirs, run_info_yaml, config, sample_names)
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 696, in _run_info_from_yaml
    item = _normalize_files(item, dirs.get("flowcell"))
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 618, in _normalize_files
    files = [_file_to_abs(x, [os.getcwd(), fc_dir, fastq_dir]) for x in files]
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 592, in _file_to_abs
    if x is None or os.path.isabs(x):
  File "/home/hodl/local/share/bcbio/anaconda/lib/python2.7/posixpath.py", line 54, in isabs
    return s.startswith('/')
AttributeError: 'list' object has no attribute 'startswith'

My config file for 1 sample looks like this (formatting is off)
- algorithm:
    aligner: bwa
    recalibrate: gatk
    realign: gatk
    ensemble:
      numpass: 1
    variant_regions: /home/hodl/references/bedfiles/Nimblegen_SeqCap_EZ_Exome_v3_primary.hs37.bed
    svcaller: [lumpy, manta, delly]
  analysis: variant2
  description: T180_baseline
  files:
  - [/home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R1.fastq.gz, /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R2.fastq.gz]
  genome_build: GRCh37
  metadata:
    batch: 180_baseline
    phenotype: tumor

Could you help me pls? Thank you!
Cannot read input for paired reads?
issue
created
chapmanb
username_1
It looks like the fastq files are formatted in your configuration as a list of lists (the `-` starts a list). You'd either want to remove the `-` and move the list up one, or do:
```
files:
  - /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R1.fastq.gz, 
 - /home/hodl/storage3/hodl/Projects/180/180_baseline/HS002-PE-R00170_BC6LRBACXX.CHC379_GCCAAT_L004_R2.fastq.gz]
```
Hope this gets things running for you.
comment
closed
chapmanb
username_1
issue