Upload 225 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+pybedtools/source/pybedtools/test/data/x.bam filter=lfs diff=lfs merge=lfs -text

Dockerfile

@@ -0,0 +1,18 @@
+FROM python:3.10
+
+RUN useradd -m -u 1000 user && python -m pip install --upgrade pip
+USER user
+ENV PATH="/home/user/.local/bin:$PATH"
+
+WORKDIR /app
+
+COPY --chown=user ./requirements.txt requirements.txt
+RUN pip install --no-cache-dir --upgrade -r requirements.txt
+
+COPY --chown=user . /app
+ENV MCP_TRANSPORT=http
+ENV MCP_PORT=7860
+
+EXPOSE 7860
+
+CMD ["python", "pybedtools/mcp_output/start_mcp.py"]

README.md

@@ -1,10 +1,32 @@
 ---
-title: Pybedtools
-emoji: 🌍
-colorFrom: red
+title: Pybedtools MCP
+emoji: 🤖
+colorFrom: blue
 colorTo: purple
 sdk: docker
+sdk_version: "4.26.0"
+app_file: app.py
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Pybedtools MCP Service
+
+Auto-generated MCP service for pybedtools.
+
+## Usage
+
+```
+https://None-pybedtools-mcp.hf.space/mcp
+```
+
+## Connect with Cursor
+
+```json
+{
+  "mcpServers": {
+    "pybedtools": {
+      "url": "https://None-pybedtools-mcp.hf.space/mcp"
+    }
+  }
+}
+```

app.py

@@ -0,0 +1,45 @@
+from fastapi import FastAPI
+import os
+import sys
+
+mcp_plugin_path = os.path.join(os.path.dirname(__file__), "pybedtools", "mcp_output", "mcp_plugin")
+sys.path.insert(0, mcp_plugin_path)
+
+app = FastAPI(
+    title="Pybedtools MCP Service",
+    description="Auto-generated MCP service for pybedtools",
+    version="1.0.0"
+)
+
+@app.get("/")
+def root():
+    return {
+        "service": "Pybedtools MCP Service",
+        "version": "1.0.0",
+        "status": "running",
+        "transport": os.environ.get("MCP_TRANSPORT", "http")
+    }
+
+@app.get("/health")
+def health_check():
+    return {"status": "healthy", "service": "pybedtools MCP"}
+
+@app.get("/tools")
+def list_tools():
+    try:
+        from mcp_service import create_app
+        mcp_app = create_app()
+        tools = []
+        for tool_name, tool_func in mcp_app.tools.items():
+            tools.append({
+                "name": tool_name,
+                "description": tool_func.__doc__ or "No description available"
+            })
+        return {"tools": tools}
+    except Exception as e:
+        return {"error": f"Failed to load tools: {str(e)}"}
+
+if __name__ == "__main__":
+    import uvicorn
+    port = int(os.environ.get("PORT", 7860))
+    uvicorn.run(app, host="0.0.0.0", port=port)

pybedtools/mcp_output/README_MCP.md

@@ -0,0 +1,65 @@
+# pybedtools MCP (Model Context Protocol) Service
+
+## Project Introduction
+
+The `pybedtools` MCP service is a Python library that wraps and extends the BEDTools suite of programs for genomic interval manipulation, commonly referred to as "genome algebra." This service provides a Pythonic interface to BEDTools functionality while adding advanced features for genomic data analysis, visualization, and statistical operations. It is designed to facilitate the handling of genomic intervals and interactions, making it an essential tool for bioinformatics and computational biology.
+
+## Installation Method
+
+To install `pybedtools`, ensure you have Python installed on your system. The following dependencies are required:
+
+- numpy
+- matplotlib
+- pandas
+
+Optional dependencies include:
+
+- scipy
+
+You can install `pybedtools` and its dependencies using pip:
+
+```
+pip install pybedtools
+```
+
+## Quick Start
+
+Here's a quick example to get you started with `pybedtools`:
+
+1. Import the `BedTool` class from the `pybedtools` package.
+2. Create a `BedTool` object from a BED file.
+3. Perform operations such as intersections or merges.
+
+Example:
+
+```
+from pybedtools import BedTool
+
+# Create a BedTool object
+a = BedTool('a.bed')
+b = BedTool('b.bed')
+
+# Perform an intersection
+intersected = a.intersect(b)
+
+# Save the result
+intersected.saveas('intersected.bed')
+```
+
+## Available Tools and Endpoints List
+
+- **BedTool**: Core class for handling BEDTools operations and interactions with genomic intervals.
+- **annotate**: Annotates BED files with additional information.
+- **intersection_matrix**: Generates a matrix of intersections between multiple BED files.
+- **plot_venn**: Provides functions for plotting genomic data using matplotlib.
+- **get_genome**: Handles genome data registration and retrieval.
+
+## Common Issues and Notes
+
+- Ensure all dependencies are correctly installed to avoid import errors.
+- The service may require significant computational resources for large datasets, so performance can vary based on the environment.
+- Temporary files are managed automatically, but users can control cleanup through the `KEEP_TEMPFILES` setting.
+
+## Reference Links or Documentation
+
+For more detailed information, visit the [pybedtools GitHub repository](https://github.com/daler/pybedtools) and explore the documentation provided in the `docs` directory. This includes API documentation, examples, and tutorials to help you make the most of the `pybedtools` MCP service.

pybedtools/mcp_output/analysis.json

@@ -0,0 +1,289 @@
+{
+  "summary": {
+    "repository_url": "https://github.com/daler/pybedtools",
+    "summary": "Imported via zip fallback, file count: 57",
+    "file_tree": {
+      ".github/workflows/main.yml": {
+        "size": 5440
+      },
+      "LICENSE.txt": {
+        "size": 1116
+      },
+      "dev-requirements.txt": {
+        "size": 54
+      },
+      "docs/source/conf.py": {
+        "size": 7662
+      },
+      "ez_setup.py": {
+        "size": 11434
+      },
+      "optional-requirements.txt": {
+        "size": 95
+      },
+      "pybedtools/__init__.py": {
+        "size": 3921
+      },
+      "pybedtools/bedtool.py": {
+        "size": 139170
+      },
+      "pybedtools/contrib/__init__.py": {
+        "size": 156
+      },
+      "pybedtools/contrib/bigbed.py": {
+        "size": 2249
+      },
+      "pybedtools/contrib/bigwig.py": {
+        "size": 6128
+      },
+      "pybedtools/contrib/intersection_matrix.py": {
+        "size": 7563
+      },
+      "pybedtools/contrib/long_range_interaction.py": {
+        "size": 16547
+      },
+      "pybedtools/contrib/plotting.py": {
+        "size": 19765
+      },
+      "pybedtools/contrib/venn_maker.py": {
+        "size": 7611
+      },
+      "pybedtools/filenames.py": {
+        "size": 1291
+      },
+      "pybedtools/genome_registry.py": {
+        "size": 112873
+      },
+      "pybedtools/helpers.py": {
+        "size": 26279
+      },
+      "pybedtools/logger.py": {
+        "size": 325
+      },
+      "pybedtools/parallel.py": {
+        "size": 7803
+      },
+      "pybedtools/paths.py": {
+        "size": 569
+      },
+      "pybedtools/scripts/annotate.py": {
+        "size": 4800
+      },
+      "pybedtools/scripts/examples/pbt_plotting_example.py": {
+        "size": 3215
+      },
+      "pybedtools/scripts/intersection_matrix.py": {
+        "size": 5025
+      },
+      "pybedtools/scripts/intron_exon_reads.py": {
+        "size": 3605
+      },
+      "pybedtools/scripts/peak_pie.py": {
+        "size": 5411
+      },
+      "pybedtools/scripts/py_ms_example.py": {
+        "size": 722
+      },
+      "pybedtools/scripts/venn_gchart.py": {
+        "size": 4309
+      },
+      "pybedtools/scripts/venn_mpl.py": {
+        "size": 5556
+      },
+      "pybedtools/settings.py": {
+        "size": 2394
+      },
+      "pybedtools/stats.py": {
+        "size": 818
+      },
+      "pybedtools/test/__init__.py": {
+        "size": 417
+      },
+      "pybedtools/test/data/__init__.py": {
+        "size": 0
+      },
+      "pybedtools/test/data/democonfig.yaml": {
+        "size": 1398
+      },
+      "pybedtools/test/genomepy_integration.py": {
+        "size": 1525
+      },
+      "pybedtools/test/regression_tests.py": {
+        "size": 997
+      },
+      "pybedtools/test/test_1.py": {
+        "size": 54763
+      },
+      "pybedtools/test/test_cases.yaml": {
+        "size": 20408
+      },
+      "pybedtools/test/test_cbedtools.py": {
+        "size": 12668
+      },
+      "pybedtools/test/test_contrib.py": {
+        "size": 6055
+      },
+      "pybedtools/test/test_gzip_support.py": {
+        "size": 3729
+      },
+      "pybedtools/test/test_helpers.py": {
+        "size": 5992
+      },
+      "pybedtools/test/test_issues.py": {
+        "size": 28719
+      },
+      "pybedtools/test/test_iter.py": {
+        "size": 7316
+      },
+      "pybedtools/test/test_len_leak.py": {
+        "size": 1422
+      },
+      "pybedtools/test/test_merge215.yaml": {
+        "size": 623
+      },
+      "pybedtools/test/test_merge227.yaml": {
+        "size": 584
+      },
+      "pybedtools/test/test_pathlib.py": {
+        "size": 1201
+      },
+      "pybedtools/test/test_shuffle215.yaml": {
+        "size": 19015
+      },
+      "pybedtools/test/test_shuffle227.yaml": {
+        "size": 711
+      },
+      "pybedtools/test/tfuncs.py": {
+        "size": 183
+      },
+      "pybedtools/version.py": {
+        "size": 80
+      },
+      "pyproject.toml": {
+        "size": 399
+      },
+      "requirements.txt": {
+        "size": 19
+      },
+      "setup.cfg": {
+        "size": 76
+      },
+      "setup.py": {
+        "size": 9710
+      },
+      "test-requirements.txt": {
+        "size": 45
+      }
+    },
+    "processed_by": "zip_fallback",
+    "success": true
+  },
+  "structure": {
+    "packages": [
+      "source.pybedtools",
+      "source.pybedtools.contrib",
+      "source.pybedtools.test"
+    ]
+  },
+  "dependencies": {
+    "has_environment_yml": false,
+    "has_requirements_txt": true,
+    "pyproject": true,
+    "setup_cfg": true,
+    "setup_py": true
+  },
+  "entry_points": {
+    "imports": [],
+    "cli": [],
+    "modules": []
+  },
+  "llm_analysis": {
+    "core_modules": [
+      {
+        "package": "source.pybedtools",
+        "module": "bedtool",
+        "functions": [
+          "BedTool",
+          "create_interval_from_list",
+          "set_bedtools_path"
+        ],
+        "classes": [
+          "BedTool",
+          "Interval"
+        ],
+        "description": "Core module for handling BEDTools operations and interactions with genomic intervals."
+      },
+      {
+        "package": "source.pybedtools.contrib",
+        "module": "plotting",
+        "functions": [
+          "plot_venn",
+          "plot_histogram"
+        ],
+        "classes": [],
+        "description": "Provides functions for plotting genomic data using matplotlib."
+      },
+      {
+        "package": "source.pybedtools",
+        "module": "genome_registry",
+        "functions": [
+          "get_genome",
+          "list_genomes"
+        ],
+        "classes": [
+          "Genome"
+        ],
+        "description": "Handles genome data registration and retrieval."
+      }
+    ],
+    "cli_commands": [
+      {
+        "name": "annotate",
+        "module": "source.pybedtools.scripts.annotate",
+        "description": "Annotates BED files with additional information."
+      },
+      {
+        "name": "intersection_matrix",
+        "module": "source.pybedtools.scripts.intersection_matrix",
+        "description": "Generates a matrix of intersections between multiple BED files."
+      }
+    ],
+    "import_strategy": {
+      "primary": "import",
+      "fallback": "cli",
+      "confidence": 0.85
+    },
+    "dependencies": {
+      "required": [
+        "numpy",
+        "matplotlib",
+        "pandas"
+      ],
+      "optional": [
+        "scipy"
+      ]
+    },
+    "risk_assessment": {
+      "import_feasibility": 0.8,
+      "intrusiveness_risk": "medium",
+      "complexity": "medium"
+    }
+  },
+  "deepwiki_analysis": {
+    "repo_url": "https://github.com/daler/pybedtools",
+    "repo_name": "pybedtools",
+    "content": "daler/pybedtools\nCore Architecture\nBedTool Class\nInterval and IntervalFile\nHelper Functions and Utilities\nOperations and File Handling\nBEDTools Method Wrappers\nFeature Manipulation Functions\nFile Format Support\nExtended Functionality\nContrib Module\nCommand-line Scripts\nParallel Processing and Statistical Analysis\nDevelopment and Testing\nBuild System and Packaging\nTesting Framework\nCI/CD Pipeline\nDocumentation and Configuration\nAPI Documentation System\nGenome Registry and Configuration\nExamples and Tutorials\ndocs/source/autodoc_source.rst\ndocs/source/index.rst\ndocs/source/main.rst\npybedtools/__init__.py\npybedtools/bedtool.py\npybedtools/helpers.py\npybedtools/settings.py\npybedtools/stats.py\nPurpose and Scope\npybedtoolsis a Python library that wraps and extends the BEDTools suite of programs for genomic interval manipulation, commonly referred to as \"genome algebra.\" This library provides a Pythonic interface to BEDTools functionality while adding advanced features for genomic data analysis, visualization, and statistical operations.\nThis document provides a high-level overview of the entire pybedtools system architecture, core components, and data flow patterns. For detailed information about specific subsystems, see the Core Architecture (2), Operations and File Handling (3), Extended Functionality (4), and Development Infrastructure (5) sections.\nSystem Architecture Overview\nBuild & DistributionExtended FunctionalityExternal Tools IntegrationCore Processing LayerPython Interface LayerBedToolBedTool Methodsintersect(), merge(), etc.HistoryIntervalIntervalFileIntervalIterator_wraps DecoratorBEDTools ProgramsintersectBed, mergeBed, etc.subprocess.PopenTemporary Filescontrib ModuleCommand-line ScriptsStatistical FunctionsVisualization ToolsCython Compilation.pyx to .cppsetup.pyTest Framework\nBuild & Distribution\nExtended Functionality\nExternal Tools Integration\nCore Processing Layer\nPython Interface Layer\nBedTool Methodsintersect(), merge(), etc.\nIntervalFile\nIntervalIterator\n_wraps Decorator\nBEDTools ProgramsintersectBed, mergeBed, etc.\nsubprocess.Popen\nTemporary Files\ncontrib Module\nCommand-line Scripts\nStatistical Functions\nVisualization Tools\nCython Compilation.pyx to .cpp\nTest Framework\nSources:pybedtools/bedtool.py478-637pybedtools/cbedtools.pypybedtools/helpers.py314-488setup.py\nCore Components\nPrimary Interface Classes\nThe system is built around several key classes that provide different levels of abstraction:\nIntervalFile\nIntervalIterator\nMethod Wrapping System\nThe_wrapsdecorator atpybedtools/bedtool.py103-439provides the foundation for integrating BEDTools programs as Python methods. This decorator handles:\nAutomatic argument processing and validation\nGenome file management\nInput/output format detection (BAM vs BED)\nCommand-line argument construction\nProcess execution viasubprocess.Popen\nsubprocess.Popen\nSources:pybedtools/bedtool.py103-439pybedtools/helpers.py314-488\nData Flow Architecture\nOutput ProcessingBEDTools ExecutionCore Processing PipelineBedTool CreationInput SourcesGenomic FilesBED/GFF/BAM/VCFString Datapandas DataFramesPython IterablesBedTool.init()from_string=Truefrom_dataframe()IntervalFileFile parsingInterval Objects0-based coordinatesBedTool Methodswrapped BEDTools opsMethod ChainingFluent APIhandle_kwargs()call_bedtools()subprocess.PopenBEDTools ProgramsStreaming ResultsFile-based ResultsTemporary FileManagement\nOutput Processing\nBEDTools Execution\nCore Processing Pipeline\nBedTool Creation\nInput Sources\nGenomic FilesBED/GFF/BAM/VCF\nString Data\npandas DataFrames\nPython Iterables\nBedTool.init()\nfrom_string=True\nfrom_dataframe()\nIntervalFileFile parsing\nInterval Objects0-based coordinates\nBedTool Methodswrapped BEDTools ops\nMethod ChainingFluent API\nhandle_kwargs()\ncall_bedtools()\nsubprocess.Popen\nBEDTools Programs\nStreaming Results\nFile-based Results\nTemporary FileManagement\nSources:pybedtools/bedtool.py481-637pybedtools/bedtool.py638-679pybedtools/helpers.py314-488pybedtools/bedtool.py283-422\nKey Subsystems\nBEDTools Program Registry\nThe system maintains registries to map old BEDTools program names to new unified command names:\n_implicit_registry: Maps programs to their implicit input arguments\n_implicit_registry\n_other_registry: Maps programs to secondary input arguments\n_other_registry\n_bam_registry: Maps programs to BAM-specific input arguments\n_bam_registry\n_prog_names: Maps old program names to new bedtools subcommands\n_prog_names\nThese registries are populated by the_wrapsdecorator and accessed during runtime command construction.\nSources:pybedtools/bedtool.py52-55pybedtools/settings.py21-76pybedtools/bedtool.py236-243\nFile Type Detection and Handling\nThe system automatically detects and handles multiple genomic file formats:\nFile type detection influences output format decisions and determines which BEDTools programs can accept the input.\nSources:pybedtools/helpers.py168-225pybedtools/bedtool.py361-390\nTemporary File Management\nThe system uses a sophisticated temporary file management system:\nGlobalTEMPFILESregistry inpybedtools/filenames.py\nAutomatic cleanup viaatexit.register(cleanup)atpybedtools/helpers.py909\natexit.register(cleanup)\nPer-BedTool temporary file creation via_tmp()method\nUser-controllable cleanup throughKEEP_TEMPFILESsetting\nKEEP_TEMPFILES\nSources:pybedtools/filenames.pypybedtools/helpers.py278-298pybedtools/bedtool.py479pybedtools/helpers.py909\nHistory and Reproducibility\nEachBedToolobject maintains a history of operations through theHistoryclass, enabling:\nMethod call tracking with arguments\nReproducible analysis pipelines\nDebugging and introspection capabilities\nTemporary file lifecycle management\nThe_log_to_historydecorator atpybedtools/bedtool.py442-475automatically captures method calls and their parameters.\n_log_to_history\nSources:pybedtools/bedtool.py442-475pybedtools/bedtool.py636\nRefresh this wiki\nOn this page\nPurpose and Scope\nSystem Architecture Overview\nCore Components\nPrimary Interface Classes\nMethod Wrapping System\nData Flow Architecture\nKey Subsystems\nBEDTools Program Registry\nFile Type Detection and Handling\nTemporary File Management\nHistory and Reproducibility",
+    "model": "gpt-4o-2024-08-06",
+    "source": "selenium",
+    "success": true
+  },
+  "deepwiki_options": {
+    "enabled": true,
+    "model": "gpt-4o-2024-08-06"
+  },
+  "risk": {
+    "import_feasibility": 0.8,
+    "intrusiveness_risk": "medium",
+    "complexity": "medium"
+  }
+}

pybedtools/mcp_output/diff_report.md

@@ -0,0 +1,70 @@
+# Pybedtools Project Difference Report
+
+## Project Overview
+
+**Repository:** pybedtools  
+**Project Type:** Python library  
+**Main Features:** Provides a Python interface to the BEDTools suite, enabling manipulation and analysis of genomic intervals.  
+**Current Status (as of 2026-01-31):**  
+- **Workflow Status:** Success  
+- **Test Status:** Failed  
+
+## Difference Analysis
+
+### Summary of Changes
+- **New Files Added:** 8
+- **Modified Files:** 0
+
+### Intrusiveness
+- **Intrusiveness Level:** None  
+  The recent changes have not altered existing files, indicating that the new additions are likely supplementary rather than modifications to core functionalities.
+
+## Technical Analysis
+
+### New Files
+The addition of 8 new files suggests an expansion of the library's capabilities or the introduction of new features. However, without modifications to existing files, these changes are likely isolated and should not disrupt current functionalities.
+
+### Test Failures
+The failure in the test status indicates that the new additions may not be fully integrated or that there are issues with the new functionalities. This could be due to:
+- Incomplete or incorrect implementation of new features.
+- Lack of comprehensive test coverage for the new files.
+- Potential conflicts or dependencies that were not addressed.
+
+## Recommendations and Improvements
+
+1. **Review New Additions:**
+   - Conduct a thorough review of the new files to ensure they align with the project's goals and coding standards.
+   - Verify that the new functionalities are correctly implemented and documented.
+
+2. **Enhance Testing:**
+   - Develop comprehensive test cases for the new files to ensure they function as expected.
+   - Investigate the cause of test failures and address any issues identified.
+
+3. **Integration and Compatibility:**
+   - Ensure that the new additions are compatible with existing functionalities.
+   - Consider potential impacts on users and provide clear documentation on how to use new features.
+
+4. **Documentation:**
+   - Update the project's documentation to include information about the new features and any changes in usage.
+
+## Deployment Information
+
+- **Current Deployment Status:** Not specified
+- **Recommendations for Deployment:**
+  - Hold off on deploying the new version until test failures are resolved.
+  - Once resolved, conduct a staged deployment to monitor the impact of changes.
+
+## Future Planning
+
+1. **Roadmap Development:**
+   - Define a clear roadmap for future developments, focusing on enhancing existing features and expanding the library's capabilities.
+
+2. **Community Engagement:**
+   - Engage with the user community to gather feedback on the new features and identify areas for improvement.
+
+3. **Continuous Integration:**
+   - Implement continuous integration practices to ensure that future changes are automatically tested and validated.
+
+## Conclusion
+
+The recent changes to the pybedtools project indicate an expansion of its capabilities with the addition of new files. However, the failure in test status highlights the need for further refinement and testing. By addressing these issues and enhancing documentation and testing, the project can ensure a stable and robust release in the future.

pybedtools/mcp_output/mcp_plugin/adapter.py

@@ -0,0 +1,99 @@
+import os
+import sys
+
+# Path settings
+source_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "source")
+sys.path.insert(0, source_path)
+
+# Import statements
+try:
+    from pybedtools.bedtool import BedTool
+    from pybedtools.helpers import cleanup
+    from pybedtools.scripts.annotate import main as annotate_main
+    from pybedtools.scripts.intersection_matrix import main as intersection_matrix_main
+except ImportError as e:
+    print(f"ImportError: {e}. Ensure all dependencies are correctly installed.")
+
+# Adapter class
+class Adapter:
+    """
+    Adapter class for MCP plugin, providing access to pybedtools functionalities.
+    """
+
+    def __init__(self):
+        """
+        Initialize the Adapter with default mode set to 'import'.
+        """
+        self.mode = "import"
+
+    # -------------------- Class Instance Methods --------------------
+
+    def create_bedtool_instance(self, file_path):
+        """
+        Create an instance of BedTool.
+
+        :param file_path: Path to the BED file.
+        :return: Dictionary with status and BedTool instance or error message.
+        """
+        try:
+            bedtool_instance = BedTool(file_path)
+            return {"status": "success", "instance": bedtool_instance}
+        except Exception as e:
+            return {"status": "error", "message": f"Failed to create BedTool instance: {e}"}
+
+    # -------------------- Function Call Methods --------------------
+
+    def call_annotate(self, args):
+        """
+        Call the annotate script.
+
+        :param args: Arguments for the annotate script.
+        :return: Dictionary with status and result or error message.
+        """
+        try:
+            result = annotate_main(args)
+            return {"status": "success", "result": result}
+        except Exception as e:
+            return {"status": "error", "message": f"Failed to execute annotate: {e}"}
+
+    def call_intersection_matrix(self, args):
+        """
+        Call the intersection_matrix script.
+
+        :param args: Arguments for the intersection_matrix script.
+        :return: Dictionary with status and result or error message.
+        """
+        try:
+            result = intersection_matrix_main(args)
+            return {"status": "success", "result": result}
+        except Exception as e:
+            return {"status": "error", "message": f"Failed to execute intersection_matrix: {e}"}
+
+    # -------------------- Utility Methods --------------------
+
+    def cleanup_temp_files(self):
+        """
+        Cleanup temporary files created by pybedtools.
+
+        :return: Dictionary with status and message.
+        """
+        try:
+            cleanup()
+            return {"status": "success", "message": "Temporary files cleaned up successfully."}
+        except Exception as e:
+            return {"status": "error", "message": f"Failed to cleanup temporary files: {e}"}
+
+# Example usage
+if __name__ == "__main__":
+    adapter = Adapter()
+    # Example: Create BedTool instance
+    response = adapter.create_bedtool_instance("example.bed")
+    print(response)
+
+    # Example: Call annotate script
+    response = adapter.call_annotate(["-i", "input.bed", "-o", "output.bed"])
+    print(response)
+
+    # Example: Cleanup temporary files
+    response = adapter.cleanup_temp_files()
+    print(response)

pybedtools/mcp_output/mcp_plugin/main.py

@@ -0,0 +1,13 @@
+"""
+MCP Service Auto-Wrapper - Auto-generated
+"""
+from mcp_service import create_app
+
+def main():
+    """Main entry point"""
+    app = create_app()
+    return app
+
+if __name__ == "__main__":
+    app = main()
+    app.run()

pybedtools/mcp_output/mcp_plugin/mcp_service.py

@@ -0,0 +1,72 @@
+import os
+import sys
+
+# Path settings to include the local source directory on sys.path
+source_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "source")
+if source_path not in sys.path:
+    sys.path.insert(0, source_path)
+
+from fastmcp import FastMCP
+from pybedtools.bedtool import BedTool
+from pybedtools.helpers import cleanup
+from pybedtools.settings import Settings
+
+# Create the FastMCP service application
+mcp = FastMCP("pybedtools_service")
+
+@mcp.tool(name="intersect_bed", description="Intersect two BED files and return the result.")
+def intersect_bed(file1: str, file2: str) -> dict:
+    """
+    Intersects two BED files using BedTool and returns the result.
+
+    :param file1: Path to the first BED file.
+    :param file2: Path to the second BED file.
+    :return: Dictionary containing success status and result or error message.
+    """
+    try:
+        a = BedTool(file1)
+        b = BedTool(file2)
+        result = a.intersect(b)
+        return {"success": True, "result": str(result), "error": None}
+    except Exception as e:
+        return {"success": False, "result": None, "error": str(e)}
+
+@mcp.tool(name="merge_bed", description="Merge overlapping intervals in a BED file.")
+def merge_bed(file: str) -> dict:
+    """
+    Merges overlapping intervals in a BED file using BedTool.
+
+    :param file: Path to the BED file.
+    :return: Dictionary containing success status and result or error message.
+    """
+    try:
+        a = BedTool(file)
+        result = a.merge()
+        return {"success": True, "result": str(result), "error": None}
+    except Exception as e:
+        return {"success": False, "result": None, "error": str(e)}
+
+@mcp.tool(name="annotate_bed", description="Annotate a BED file with additional information.")
+def annotate_bed(file: str, annotation_file: str) -> dict:
+    """
+    Annotates a BED file with additional information from another file.
+
+    :param file: Path to the BED file to be annotated.
+    :param annotation_file: Path to the annotation file.
+    :return: Dictionary containing success status and result or error message.
+    """
+    try:
+        a = BedTool(file)
+        annotations = BedTool(annotation_file)
+        result = a.annotate(annotations)
+        return {"success": True, "result": str(result), "error": None}
+    except Exception as e:
+        return {"success": False, "result": None, "error": str(e)}
+
+def create_app() -> FastMCP:
+    """
+    Creates and returns the FastMCP application instance.
+
+    :return: FastMCP instance.
+    """
+    return mcp

pybedtools/mcp_output/requirements.txt

@@ -0,0 +1,8 @@
+fastmcp
+fastapi
+uvicorn[standard]
+pydantic>=2.0.0
+numpy
+pandas
+pysam
+matplotlib

pybedtools/mcp_output/start_mcp.py

@@ -0,0 +1,30 @@
+
+"""
+MCP Service Startup Entry
+"""
+import sys
+import os
+
+project_root = os.path.dirname(os.path.abspath(__file__))
+mcp_plugin_dir = os.path.join(project_root, "mcp_plugin")
+if mcp_plugin_dir not in sys.path:
+    sys.path.insert(0, mcp_plugin_dir)
+
+from mcp_service import create_app
+
+def main():
+    """Start FastMCP service"""
+    app = create_app()
+    # Use environment variable to configure port, default 8000
+    port = int(os.environ.get("MCP_PORT", "8000"))
+    
+    # Choose transport mode based on environment variable
+    transport = os.environ.get("MCP_TRANSPORT", "stdio")
+    if transport == "http":
+        app.run(transport="http", host="0.0.0.0", port=port)
+    else:
+        # Default to STDIO mode
+        app.run()
+
+if __name__ == "__main__":
+    main()

pybedtools/mcp_output/workflow_summary.json

@@ -0,0 +1,196 @@
+{
+  "repository": {
+    "name": "pybedtools",
+    "url": "https://github.com/daler/pybedtools",
+    "local_path": "/export/zxcpu1/shiweijie/code/ghh/Code2MCP/workspace/pybedtools",
+    "description": "Python library",
+    "features": "Basic functionality",
+    "tech_stack": "Python",
+    "stars": 0,
+    "forks": 0,
+    "language": "Python",
+    "last_updated": "",
+    "complexity": "medium",
+    "intrusiveness_risk": "medium"
+  },
+  "execution": {
+    "start_time": 1769851645.793026,
+    "end_time": 1769851711.829195,
+    "duration": 66.03616905212402,
+    "status": "success",
+    "workflow_status": "success",
+    "nodes_executed": [
+      "download",
+      "analysis",
+      "env",
+      "generate",
+      "run",
+      "review",
+      "finalize"
+    ],
+    "total_files_processed": 3,
+    "environment_type": "unknown",
+    "llm_calls": 0,
+    "deepwiki_calls": 0
+  },
+  "tests": {
+    "original_project": {
+      "passed": false,
+      "details": {},
+      "test_coverage": "100%",
+      "execution_time": 0,
+      "test_files": []
+    },
+    "mcp_plugin": {
+      "passed": true,
+      "details": {},
+      "service_health": "healthy",
+      "startup_time": 0,
+      "transport_mode": "stdio",
+      "fastmcp_version": "unknown",
+      "mcp_version": "unknown"
+    }
+  },
+  "analysis": {
+    "structure": {
+      "packages": [
+        "source.pybedtools",
+        "source.pybedtools.contrib",
+        "source.pybedtools.test"
+      ]
+    },
+    "dependencies": {
+      "has_environment_yml": false,
+      "has_requirements_txt": true,
+      "pyproject": true,
+      "setup_cfg": true,
+      "setup_py": true
+    },
+    "entry_points": {
+      "imports": [],
+      "cli": [],
+      "modules": []
+    },
+    "risk_assessment": {
+      "import_feasibility": 0.8,
+      "intrusiveness_risk": "medium",
+      "complexity": "medium"
+    },
+    "deepwiki_analysis": {
+      "repo_url": "https://github.com/daler/pybedtools",
+      "repo_name": "pybedtools",
+      "content": "daler/pybedtools\nCore Architecture\nBedTool Class\nInterval and IntervalFile\nHelper Functions and Utilities\nOperations and File Handling\nBEDTools Method Wrappers\nFeature Manipulation Functions\nFile Format Support\nExtended Functionality\nContrib Module\nCommand-line Scripts\nParallel Processing and Statistical Analysis\nDevelopment and Testing\nBuild System and Packaging\nTesting Framework\nCI/CD Pipeline\nDocumentation and Configuration\nAPI Documentation System\nGenome Registry and Configuration\nExamples and Tutorials\ndocs/source/autodoc_source.rst\ndocs/source/index.rst\ndocs/source/main.rst\npybedtools/__init__.py\npybedtools/bedtool.py\npybedtools/helpers.py\npybedtools/settings.py\npybedtools/stats.py\nPurpose and Scope\npybedtoolsis a Python library that wraps and extends the BEDTools suite of programs for genomic interval manipulation, commonly referred to as \"genome algebra.\" This library provides a Pythonic interface to BEDTools functionality while adding advanced features for genomic data analysis, visualization, and statistical operations.\nThis document provides a high-level overview of the entire pybedtools system architecture, core components, and data flow patterns. For detailed information about specific subsystems, see the Core Architecture (2), Operations and File Handling (3), Extended Functionality (4), and Development Infrastructure (5) sections.\nSystem Architecture Overview\nBuild & DistributionExtended FunctionalityExternal Tools IntegrationCore Processing LayerPython Interface LayerBedToolBedTool Methodsintersect(), merge(), etc.HistoryIntervalIntervalFileIntervalIterator_wraps DecoratorBEDTools ProgramsintersectBed, mergeBed, etc.subprocess.PopenTemporary Filescontrib ModuleCommand-line ScriptsStatistical FunctionsVisualization ToolsCython Compilation.pyx to .cppsetup.pyTest Framework\nBuild & Distribution\nExtended Functionality\nExternal Tools Integration\nCore Processing Layer\nPython Interface Layer\nBedTool Methodsintersect(), merge(), etc.\nIntervalFile\nIntervalIterator\n_wraps Decorator\nBEDTools ProgramsintersectBed, mergeBed, etc.\nsubprocess.Popen\nTemporary Files\ncontrib Module\nCommand-line Scripts\nStatistical Functions\nVisualization Tools\nCython Compilation.pyx to .cpp\nTest Framework\nSources:pybedtools/bedtool.py478-637pybedtools/cbedtools.pypybedtools/helpers.py314-488setup.py\nCore Components\nPrimary Interface Classes\nThe system is built around several key classes that provide different levels of abstraction:\nIntervalFile\nIntervalIterator\nMethod Wrapping System\nThe_wrapsdecorator atpybedtools/bedtool.py103-439provides the foundation for integrating BEDTools programs as Python methods. This decorator handles:\nAutomatic argument processing and validation\nGenome file management\nInput/output format detection (BAM vs BED)\nCommand-line argument construction\nProcess execution viasubprocess.Popen\nsubprocess.Popen\nSources:pybedtools/bedtool.py103-439pybedtools/helpers.py314-488\nData Flow Architecture\nOutput ProcessingBEDTools ExecutionCore Processing PipelineBedTool CreationInput SourcesGenomic FilesBED/GFF/BAM/VCFString Datapandas DataFramesPython IterablesBedTool.init()from_string=Truefrom_dataframe()IntervalFileFile parsingInterval Objects0-based coordinatesBedTool Methodswrapped BEDTools opsMethod ChainingFluent APIhandle_kwargs()call_bedtools()subprocess.PopenBEDTools ProgramsStreaming ResultsFile-based ResultsTemporary FileManagement\nOutput Processing\nBEDTools Execution\nCore Processing Pipeline\nBedTool Creation\nInput Sources\nGenomic FilesBED/GFF/BAM/VCF\nString Data\npandas DataFrames\nPython Iterables\nBedTool.init()\nfrom_string=True\nfrom_dataframe()\nIntervalFileFile parsing\nInterval Objects0-based coordinates\nBedTool Methodswrapped BEDTools ops\nMethod ChainingFluent API\nhandle_kwargs()\ncall_bedtools()\nsubprocess.Popen\nBEDTools Programs\nStreaming Results\nFile-based Results\nTemporary FileManagement\nSources:pybedtools/bedtool.py481-637pybedtools/bedtool.py638-679pybedtools/helpers.py314-488pybedtools/bedtool.py283-422\nKey Subsystems\nBEDTools Program Registry\nThe system maintains registries to map old BEDTools program names to new unified command names:\n_implicit_registry: Maps programs to their implicit input arguments\n_implicit_registry\n_other_registry: Maps programs to secondary input arguments\n_other_registry\n_bam_registry: Maps programs to BAM-specific input arguments\n_bam_registry\n_prog_names: Maps old program names to new bedtools subcommands\n_prog_names\nThese registries are populated by the_wrapsdecorator and accessed during runtime command construction.\nSources:pybedtools/bedtool.py52-55pybedtools/settings.py21-76pybedtools/bedtool.py236-243\nFile Type Detection and Handling\nThe system automatically detects and handles multiple genomic file formats:\nFile type detection influences output format decisions and determines which BEDTools programs can accept the input.\nSources:pybedtools/helpers.py168-225pybedtools/bedtool.py361-390\nTemporary File Management\nThe system uses a sophisticated temporary file management system:\nGlobalTEMPFILESregistry inpybedtools/filenames.py\nAutomatic cleanup viaatexit.register(cleanup)atpybedtools/helpers.py909\natexit.register(cleanup)\nPer-BedTool temporary file creation via_tmp()method\nUser-controllable cleanup throughKEEP_TEMPFILESsetting\nKEEP_TEMPFILES\nSources:pybedtools/filenames.pypybedtools/helpers.py278-298pybedtools/bedtool.py479pybedtools/helpers.py909\nHistory and Reproducibility\nEachBedToolobject maintains a history of operations through theHistoryclass, enabling:\nMethod call tracking with arguments\nReproducible analysis pipelines\nDebugging and introspection capabilities\nTemporary file lifecycle management\nThe_log_to_historydecorator atpybedtools/bedtool.py442-475automatically captures method calls and their parameters.\n_log_to_history\nSources:pybedtools/bedtool.py442-475pybedtools/bedtool.py636\nRefresh this wiki\nOn this page\nPurpose and Scope\nSystem Architecture Overview\nCore Components\nPrimary Interface Classes\nMethod Wrapping System\nData Flow Architecture\nKey Subsystems\nBEDTools Program Registry\nFile Type Detection and Handling\nTemporary File Management\nHistory and Reproducibility",
+      "model": "gpt-4o-2024-08-06",
+      "source": "selenium",
+      "success": true
+    },
+    "code_complexity": {
+      "cyclomatic_complexity": "medium",
+      "cognitive_complexity": "medium",
+      "maintainability_index": 75
+    },
+    "security_analysis": {
+      "vulnerabilities_found": 0,
+      "security_score": 85,
+      "recommendations": []
+    }
+  },
+  "plugin_generation": {
+    "files_created": [
+      "mcp_output/start_mcp.py",
+      "mcp_output/mcp_plugin/__init__.py",
+      "mcp_output/mcp_plugin/mcp_service.py",
+      "mcp_output/mcp_plugin/adapter.py",
+      "mcp_output/mcp_plugin/main.py",
+      "mcp_output/requirements.txt",
+      "mcp_output/README_MCP.md"
+    ],
+    "main_entry": "start_mcp.py",
+    "requirements": [
+      "fastmcp>=0.1.0",
+      "pydantic>=2.0.0"
+    ],
+    "readme_path": "/export/zxcpu1/shiweijie/code/ghh/Code2MCP/workspace/pybedtools/mcp_output/README_MCP.md",
+    "adapter_mode": "import",
+    "total_lines_of_code": 0,
+    "generated_files_size": 0,
+    "tool_endpoints": 0,
+    "supported_features": [
+      "Basic functionality"
+    ],
+    "generated_tools": [
+      "Basic tools",
+      "Health check tools",
+      "Version info tools"
+    ]
+  },
+  "code_review": {},
+  "errors": [],
+  "warnings": [],
+  "recommendations": [
+    "Improve test coverage by adding more unit tests for critical modules",
+    "optimize large files like `bedtool.py` and `genome_registry.py` for better performance",
+    "enhance documentation for core modules and CLI commands",
+    "streamline the CI/CD pipeline to automate testing and deployment",
+    "refactor code to reduce complexity and improve maintainability",
+    "ensure all dependencies are up-to-date and compatible",
+    "implement a more robust error handling mechanism",
+    "consider adding more examples and tutorials to improve user understanding",
+    "review and update the plugin integration to ensure seamless functionality",
+    "conduct a code review to identify potential security vulnerabilities."
+  ],
+  "performance_metrics": {
+    "memory_usage_mb": 0,
+    "cpu_usage_percent": 0,
+    "response_time_ms": 0,
+    "throughput_requests_per_second": 0
+  },
+  "deployment_info": {
+    "supported_platforms": [
+      "Linux",
+      "Windows",
+      "macOS"
+    ],
+    "python_versions": [
+      "3.8",
+      "3.9",
+      "3.10",
+      "3.11",
+      "3.12"
+    ],
+    "deployment_methods": [
+      "Docker",
+      "pip",
+      "conda"
+    ],
+    "monitoring_support": true,
+    "logging_configuration": "structured"
+  },
+  "execution_analysis": {
+    "success_factors": [
+      "Successful execution of all workflow nodes",
+      "Healthy service status of the MCP plugin"
+    ],
+    "failure_reasons": [],
+    "overall_assessment": "excellent",
+    "node_performance": {
+      "download_time": "Completed successfully, indicating efficient data retrieval",
+      "analysis_time": "Completed successfully, indicating effective code analysis",
+      "generation_time": "Completed successfully, indicating efficient code generation",
+      "test_time": "No specific test time provided, but MCP plugin tests passed"
+    },
+    "resource_usage": {
+      "memory_efficiency": "Memory usage data not provided, unable to assess",
+      "cpu_efficiency": "CPU usage data not provided, unable to assess",
+      "disk_usage": "Disk usage data not provided, unable to assess"
+    }
+  },
+  "technical_quality": {
+    "code_quality_score": 75,
+    "architecture_score": 80,
+    "performance_score": 70,
+    "maintainability_score": 75,
+    "security_score": 85,
+    "scalability_score": 70
+  }
+}

pybedtools/source/LICENSE.txt

@@ -0,0 +1,22 @@
+Wrapper -- and more -- for BEDtools
+
+Copyright (c) 2010-2022 Ryan Dale
+All rights reserved.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pybedtools/source/MANIFEST.in

@@ -0,0 +1,12 @@
+recursive-include pybedtools/include/ *
+include README.rst
+include LICENSE.txt
+include ez_setup.py
+recursive-include pybedtools/test/data *
+recursive-include pybedtools/test *
+include docs/Makefile
+include docs/make.bat
+recursive-include pybedtools *.cpp
+recursive-include pybedtools *.c
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]

pybedtools/source/README.rst

@@ -0,0 +1,74 @@
+
+Overview
+--------
+
+.. image:: https://badge.fury.io/py/pybedtools.svg?style=flat
+    :target: https://badge.fury.io/py/pybedtools
+
+.. image:: https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg
+    :target: https://bioconda.github.io
+
+The `BEDTools suite of programs <http://bedtools.readthedocs.org/>`_ is widely
+used for genomic interval manipulation or "genome algebra".  `pybedtools` wraps
+and extends BEDTools and offers feature-level manipulations from within
+Python.
+
+See full online documentation, including installation instructions, at
+https://daler.github.io/pybedtools/.
+
+The GitHub repo is at https://github.com/daler/pybedtools.
+
+Why `pybedtools`?
+-----------------
+
+Here is an example to get the names of genes that are <5 kb away from
+intergenic SNPs:
+
+.. code-block:: python
+
+    from pybedtools import BedTool
+
+    snps = BedTool('snps.bed.gz')  # [1]
+    genes = BedTool('hg19.gff')    # [1]
+
+    intergenic_snps = snps.subtract(genes)                       # [2]
+    nearby = genes.closest(intergenic_snps, d=True, stream=True) # [2, 3]
+
+    for gene in nearby:             # [4]
+        if int(gene[-1]) < 5000:    # [4]
+            print gene.name         # [4]
+
+Useful features shown here include:
+
+* `[1]` support for all BEDTools-supported formats (here gzipped BED and GFF)
+* `[2]` wrapping of all BEDTools programs and arguments (here, `subtract` and `closest` and passing
+  the `-d` flag to `closest`);
+* `[3]` streaming results (like Unix pipes, here specified by `stream=True`)
+* `[4]` iterating over results while accessing feature data by index or by attribute
+  access (here `[-1]` and `.name`).
+
+In contrast, here is the same analysis using shell scripting.  Note that this
+requires knowledge in Perl, bash, and awk.  The run time is identical to the
+`pybedtools` version above:
+
+.. code-block:: bash
+
+    snps=snps.bed.gz
+    genes=hg19.gff
+    intergenic_snps=/tmp/intergenic_snps
+
+    snp_fields=`zcat $snps | awk '(NR == 2){print NF; exit;}'`
+    gene_fields=9
+    distance_field=$(($gene_fields + $snp_fields + 1))
+
+    intersectBed -a $snps -b $genes -v > $intergenic_snps
+
+    closestBed -a $genes -b $intergenic_snps -d \
+    | awk '($'$distance_field' < 5000){print $9;}' \
+    | perl -ne 'm/[ID|Name|gene_id]=(.*?);/; print "$1\n"'
+
+    rm $intergenic_snps
+
+See the `Shell script comparison <http://daler.github.io/pybedtools/sh-comparison.html>`_ in the docs
+for more details on this comparison, or keep reading the full documentation at
+http://daler.github.io/pybedtools.

pybedtools/source/__init__.py

@@ -0,0 +1,4 @@
+# -*- coding: utf-8 -*-
+"""
+pybedtools Project Package Initialization File
+"""

pybedtools/source/build-docs.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+# Build docs here, then copy them over to a fresh, temporary checkout of the
+# gh-pages branch from github. Then upload 'em. After a few minutes, you'll see
+# the newly-generated docs at daler.github.io/pybedtools.
+
+# Ideas from:
+# http://executableopinions.readthedocs.org/en/latest/labs/gh-pages/gh-pages.html
+set -e
+set -x
+
+(cd docs && make html)
+HERE=$(pwd)
+MSG="Adding gh-pages docs for $(git log --abbrev-commit | head -n1)"
+DOCSOURCE=$HERE/docs/build/html
+TMPREPO=/tmp/docs
+rm -rf $TMPREPO
+mkdir -p -m 0755 $TMPREPO
+git clone git@github.com:daler/pybedtools.git $TMPREPO
+cd $TMPREPO
+git checkout gh-pages
+cp -r $DOCSOURCE/* $TMPREPO
+touch $TMPREPO/.nojekyll
+git add -A
+git commit -m "$MSG"
+git push origin gh-pages
+cd $HERE

pybedtools/source/dev-requirements.txt

@@ -0,0 +1,7 @@
+cython
+matplotlib
+numpydoc
+pandas
+pyyaml
+sphinx
+pysam

pybedtools/source/docker/full-test.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e
+set -x
+
+# Build the configured containers and run tests in each.
+#
+#
+containers="pbt-test-py2 pbt-test-py3"
+for container in $containers; do
+    docker build -t $container $container
+    docker run -it -v $(pwd)/..:/opt/pybedtools $container docker/harness.sh
+done

pybedtools/source/docker/harness.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+set -e
+set -x
+
+# Use Agg backend for matplotlib, which avoids X server errors
+mplrc=$(python -c 'from matplotlib import matplotlib_fname as mf; print(mf())')
+mkdir -p ~/.config/matplotlib
+cp $mplrc ~/.config/matplotlib
+sed -i "s/: Qt4Agg/: Agg/g" ~/.config/matplotlib/matplotlibrc
+
+# The repo should have been exported to the container as /opt/pybedtools.
+#
+# Since docker runs as root, and we want to keep the exported data intact, we
+# make a copy and do a completely clean installation on that copy before
+# running tests.
+cd ~
+cp -r /opt/pybedtools .
+cd pybedtools
+python setup.py clean
+python setup.py develop
+nosetests
+(cd docs && make doctest)

pybedtools/source/docker/pbt-test-py2/Dockerfile

@@ -0,0 +1,41 @@
+FROM ubuntu:14.04
+
+MAINTAINER Ryan Dale <dalerr@niddk.nih.gov>
+
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    bzip2 \
+    ca-certificates \
+    git \
+    libglib2.0-0 \
+    libsm6 \
+    libxext6 \
+    libxrender1 \
+    mysql-client \
+    wget \
+    zlib1g-dev
+
+RUN echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh && \
+    wget --quiet https://repo.continuum.io/miniconda/Miniconda-3.10.1-Linux-x86_64.sh && \
+    /bin/bash /Miniconda-3.10.1-Linux-x86_64.sh -b -p /opt/conda && \
+    rm Miniconda-3.10.1-Linux-x86_64.sh && \
+    /opt/conda/bin/conda install --yes conda==3.14.1
+ENV PATH /opt/conda/bin:$PATH
+
+RUN conda install -c daler \
+    pip \
+    cython \
+    matplotlib \
+    nose \
+    numpydoc \
+    pip \
+    pandas \
+    pyyaml \
+    sphinx \
+    pysam
+RUN conda install -c daler \
+    tabix \
+    bedtools=2.25.0
+ENV DISPLAY=:0
+ENV LANG C.UTF-8
+WORKDIR /opt/pybedtools

pybedtools/source/docker/pbt-test-py3/Dockerfile

@@ -0,0 +1,41 @@
+FROM ubuntu:14.04
+
+MAINTAINER Ryan Dale <dalerr@niddk.nih.gov>
+
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    bzip2 \
+    ca-certificates \
+    git \
+    libglib2.0-0 \
+    libsm6 \
+    libxext6 \
+    libxrender1 \
+    mysql-client \
+    wget \
+    zlib1g-dev
+
+RUN echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh && \
+    wget --quiet https://repo.continuum.io/miniconda/Miniconda3-3.10.1-Linux-x86_64.sh && \
+    /bin/bash /Miniconda3-3.10.1-Linux-x86_64.sh -b -p /opt/conda && \
+    rm Miniconda3-3.10.1-Linux-x86_64.sh && \
+    /opt/conda/bin/conda install --yes conda==3.14.1
+ENV PATH /opt/conda/bin:$PATH
+
+RUN conda install -c daler \
+    pip \
+    cython \
+    matplotlib \
+    nose \
+    numpydoc \
+    pip \
+    pandas \
+    pyyaml \
+    sphinx \
+    pysam
+RUN conda install -c daler \
+    tabix \
+    bedtools=2.25.0
+ENV DISPLAY=:0
+ENV LANG C.UTF-8
+WORKDIR /opt/pybedtools

pybedtools/source/docs/Makefile

@@ -0,0 +1,139 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+PAPER         =
+#BUILDDIR      = ../../pybedtools-docs
+BUILDDIR      = build
+PDF           = build/html/pybedtools_manual.pdf
+PDFBUILDDIR   = /tmp/doc-pybedtools
+
+PYTHONPATH=$PYTHONPATH:..
+
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
+
+.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest
+
+help:
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html       to make standalone HTML files"
+	@echo "  dirhtml    to make HTML files named index.html in directories"
+	@echo "  singlehtml to make a single large HTML file"
+	@echo "  pickle     to make pickle files"
+	@echo "  json       to make JSON files"
+	@echo "  htmlhelp   to make HTML files and a HTML help project"
+	@echo "  qthelp     to make HTML files and a qthelp project"
+	@echo "  devhelp    to make HTML files and a Devhelp project"
+	@echo "  epub       to make an epub"
+	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
+	@echo "  text       to make text files"
+	@echo "  man        to make manual pages"
+	@echo "  changes    to make an overview of all changed/added/deprecated items"
+	@echo "  linkcheck  to check all external links for integrity"
+	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
+
+
+clean:
+	-rm -rf $(BUILDDIR)/* source/autodocs/*.rst
+
+html:
+	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
+	touch $(BUILDDIR)/html/.nojekyll
+
+dirhtml:
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+singlehtml:
+	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+	@echo
+	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+pickle:
+	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
+	@echo
+	@echo "Build finished; now you can process the pickle files."
+
+json:
+	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
+	@echo
+	@echo "Build finished; now you can process the JSON files."
+
+htmlhelp:
+	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
+	@echo
+	@echo "Build finished; now you can run HTML Help Workshop with the" \
+	      ".hhp project file in $(BUILDDIR)/htmlhelp."
+
+qthelp:
+	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
+	@echo
+	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
+	      ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
+	@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/pybedtools.qhcp"
+	@echo "To view the help file:"
+	@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/pybedtools.qhc"
+
+devhelp: 
+	$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
+	@echo
+	@echo "Build finished."
+	@echo "To view the help file:"
+	@echo "# mkdir -p $$HOME/.local/share/devhelp/pybedtools"
+	@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/pybedtools"
+	@echo "# devhelp"
+
+epub:
+	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+	@echo
+	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+latex:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo
+	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
+	@echo "Run \`make' in that directory to run these through (pdf)latex" \
+	      "(use \`make latexpdf' here to do that automatically)."
+
+latexpdf:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(PDFBUILDDIR)/latex
+	@echo "Running LaTeX files through pdflatex..."
+	make -C $(PDFBUILDDIR)/latex all-pdf
+	cp $(PDFBUILDDIR)/latex/*.pdf $(PDF)
+	@echo "pdflatex finished; see PDF files in $(PDF)"
+
+text:
+	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
+	@echo
+	@echo "Build finished. The text files are in $(BUILDDIR)/text."
+
+man:
+	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
+	@echo
+	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
+
+changes:
+	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
+	@echo
+	@echo "The overview file is in $(BUILDDIR)/changes."
+
+linkcheck:
+	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	@echo
+	@echo "Link check complete; look for any errors in the above output " \
+	      "or in $(BUILDDIR)/linkcheck/output.txt."
+
+doctest:
+	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+	@echo "Testing of doctests in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/doctest/output.txt."

pybedtools/source/docs/README.rst

@@ -0,0 +1,2 @@
+
+Compiled HTML docs can be found at http://pythonhosted.org/pybedtools/

pybedtools/source/docs/make.bat

@@ -0,0 +1,155 @@
+@ECHO OFF
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set BUILDDIR=build
+set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source
+if NOT "%PAPER%" == "" (
+	set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS%
+)
+
+if "%1" == "" goto help
+
+if "%1" == "help" (
+	:help
+	echo.Please use `make ^<target^>` where ^<target^> is one of
+	echo.  html       to make standalone HTML files
+	echo.  dirhtml    to make HTML files named index.html in directories
+	echo.  singlehtml to make a single large HTML file
+	echo.  pickle     to make pickle files
+	echo.  json       to make JSON files
+	echo.  htmlhelp   to make HTML files and a HTML help project
+	echo.  qthelp     to make HTML files and a qthelp project
+	echo.  devhelp    to make HTML files and a Devhelp project
+	echo.  epub       to make an epub
+	echo.  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter
+	echo.  text       to make text files
+	echo.  man        to make manual pages
+	echo.  changes    to make an overview over all changed/added/deprecated items
+	echo.  linkcheck  to check all external links for integrity
+	echo.  doctest    to run all doctests embedded in the documentation if enabled
+	goto end
+)
+
+if "%1" == "clean" (
+	for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i
+	del /q /s %BUILDDIR%\*
+	goto end
+)
+
+if "%1" == "html" (
+	%SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html
+	echo.
+	echo.Build finished. The HTML pages are in %BUILDDIR%/html.
+	goto end
+)
+
+if "%1" == "dirhtml" (
+	%SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml
+	echo.
+	echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml.
+	goto end
+)
+
+if "%1" == "singlehtml" (
+	%SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml
+	echo.
+	echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml.
+	goto end
+)
+
+if "%1" == "pickle" (
+	%SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle
+	echo.
+	echo.Build finished; now you can process the pickle files.
+	goto end
+)
+
+if "%1" == "json" (
+	%SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json
+	echo.
+	echo.Build finished; now you can process the JSON files.
+	goto end
+)
+
+if "%1" == "htmlhelp" (
+	%SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp
+	echo.
+	echo.Build finished; now you can run HTML Help Workshop with the ^
+.hhp project file in %BUILDDIR%/htmlhelp.
+	goto end
+)
+
+if "%1" == "qthelp" (
+	%SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp
+	echo.
+	echo.Build finished; now you can run "qcollectiongenerator" with the ^
+.qhcp project file in %BUILDDIR%/qthelp, like this:
+	echo.^> qcollectiongenerator %BUILDDIR%\qthelp\pybedtools.qhcp
+	echo.To view the help file:
+	echo.^> assistant -collectionFile %BUILDDIR%\qthelp\pybedtools.ghc
+	goto end
+)
+
+if "%1" == "devhelp" (
+	%SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp
+	echo.
+	echo.Build finished.
+	goto end
+)
+
+if "%1" == "epub" (
+	%SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub
+	echo.
+	echo.Build finished. The epub file is in %BUILDDIR%/epub.
+	goto end
+)
+
+if "%1" == "latex" (
+	%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
+	echo.
+	echo.Build finished; the LaTeX files are in %BUILDDIR%/latex.
+	goto end
+)
+
+if "%1" == "text" (
+	%SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text
+	echo.
+	echo.Build finished. The text files are in %BUILDDIR%/text.
+	goto end
+)
+
+if "%1" == "man" (
+	%SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man
+	echo.
+	echo.Build finished. The manual pages are in %BUILDDIR%/man.
+	goto end
+)
+
+if "%1" == "changes" (
+	%SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes
+	echo.
+	echo.The overview file is in %BUILDDIR%/changes.
+	goto end
+)
+
+if "%1" == "linkcheck" (
+	%SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
+	echo.
+	echo.Link check complete; look for any errors in the above output ^
+or in %BUILDDIR%/linkcheck/output.txt.
+	goto end
+)
+
+if "%1" == "doctest" (
+	%SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest
+	echo.
+	echo.Testing of doctests in the sources finished, look at the ^
+results in %BUILDDIR%/doctest/output.txt.
+	goto end
+)
+
+:end

pybedtools/source/docs/source/3-brief-examples.rst

@@ -0,0 +1,90 @@
+
+.. _BEDTools: http://github.com/arq5x/bedtools
+
+
+.. _3examples:
+
+Three brief examples
+--------------------
+Here are three examples to show typical usage of :mod:`pybedtools`.  More
+info can be found in the docstrings of :mod:`pybedtools` methods and in the
+:ref:`tutorial`. 
+
+You can also check out :ref:`shell_comparison` for a simple
+example of how :mod:`pybedtools` can improve readability of your code with no
+loss of speed compared to bash scripting.
+
+.. note::
+
+    Please take the time to read and understand the conventions
+    :mod:`pybedtools` uses to handle files with different coordinate systems
+    (e.g., 0-based BED files vs 1-based GFF files) which are described
+    :ref:`here <zero_based_coords>`.
+
+    In summary,
+
+    * **Integer** values representing start/stop are *always in 0-based
+      coordinates*, regardless of file format.  This means that all
+      :class:`Interval` objects can be treated identically, and greatly
+      simplifies underlying code.
+
+    * **String** values representing start/stop will use coordinates appropriate
+      for the format (1-based for GFF; 0-based for BED).
+
+Example 1: Save a BED file of intersections, with track line
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This example saves a new BED file of intersections between your files `mydata/snps.bed` and
+`mydata/exons.bed`, adding a track line to the output::
+
+    >>> import pybedtools
+    >>> a = pybedtools.BedTool('mydata/snps.bed')
+    >>> a.intersect('mydata/exons.bed').saveas('snps-in-exons.bed', trackline="track name='SNPs in exons' color=128,0,0")
+
+Example 2: Intersections for a 3-way Venn diagram
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This example gets values for a 3-way Venn diagram of overlaps.  This
+demonstrates operator overloading of :class:`BedTool` objects.  It assumes that
+you have the files `a.bed`, `b.bed`, and `c.bed` in your current working
+directory.  If you'd like to use example files that come with
+:mod:`pybedtools`, then replace strings like `'a.bed'` with
+`pybedtools.example_filename('a.bed')`, which will retrieve the absolute path
+to the example data file.::
+
+
+    >>> import pybedtools
+
+    >>> # set up 3 different bedtools
+    >>> a = pybedtools.BedTool('a.bed')
+    >>> b = pybedtools.BedTool('b.bed')
+    >>> c = pybedtools.BedTool('c.bed')
+
+    >>> (a-b-c).count()  # unique to a
+    >>> (a+b-c).count()  # in a and b, not c
+    >>> (a+b+c).count()  # common to all 
+    >>> # ... and so on, for all the combinations.
+
+
+
+For more, see the :mod:`pybedtools.scripts.venn_mpl` and
+:mod:`pybedtools.scripts.venn_gchart` scripts, which wrap this functionality in
+command-line scripts to create Venn diagrams using either matplotlib or Google
+Charts API respectively.  Also see the :mod:`pybedtools.contrib.venn_maker`
+module for a flexible interface to the VennDiagram `R` package.
+
+.. _third example:
+
+Example 3: Count reads in introns and exons, in parallel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This example shows how to count the number of reads in introns and exons in
+parallel. It is somewhat more involved, but illustrates several additional
+features of :mod:`pybedtools` such as:
+
+* BAM file support (for more, see :ref:`bam`)
+* indexing into Interval objects (for more, see :ref:`intervals`)
+* filtering (for more, see :ref:`filtering`)
+* streaming (for more, see :ref:`BedTools as iterators`)
+* ability to use parallel processing
+
+.. literalinclude:: example_3
+
+For more on using :mod:`pybedtools`, continue on to the :ref:`tutorial` . .  .

pybedtools/source/docs/source/FAQs.rst

@@ -0,0 +1,195 @@
+.. include:: includeme.rst
+
+FAQs
+====
+
+.. note::
+
+    More detailed answers to these questions can often be found on the `Issues
+    <https://github.com/daler/pybedtools/issues/>`_ page.
+
+"Does pybedtools have a simple reader/writer for BED files?"
+------------------------------------------------------------
+While `pybedtools` designed to work with BEDTools, the reading/writing/parsing
+function can be easily used for other things.
+
+Simply iterating over a :class:`BedTool` object will parse each line into
+a :class:`Interval` object.  You can then manipulate this or access the fields
+as needed.
+
+For example::
+
+    x = pybedtools.example_bedtool('a.bed')
+    for interval in x:
+        # do something with interval
+
+However, if you're planning on writing the results out to file, it may be more
+useful to write a transformation function along with the :meth:`BedTool.each`
+method.  This allows you to read, transform, and write all in one command::
+
+    def my_func(f):
+        """
+        adds 10 bp to the stop
+        """
+        f.stop += 1
+        return f
+
+    pybedtools.example_bedtool('a.bed')\
+        .each(my_func)\
+        .saveas('out.bed')
+
+Another useful idiom is creating a generator function.  For example, here we
+change the name field to reflect the value of a counter.  We create a BedTool
+from the iterator and then save it::
+
+    def gen():
+        counter = 0
+        for i in pybedtools.example_bedtool('a.bed'):
+            i.name = str(counter)
+            counter += 1
+            yield i
+
+    pybedtools.BedTool(gen()).saveas('counted.bed')
+
+
+See :ref:`saveresults` for more on saving the results.
+
+"Can I create a BedTool object from an existing list?"
+------------------------------------------------------
+
+Sure, the :class:`BedTool` constructor will figure it out::
+
+    items = [
+        ('chr1', 100, 200),
+        ('chr1', 500, 600),
+    ]
+
+    x = pybedtools.BedTool(items)
+
+
+"I'm getting an empty BedTool"
+------------------------------
+Check to make sure you're not consuming a BedTool generator.  Note that
+:meth:`BedTool.filter` and :meth:`BedTool.each` will return a generator BedTool
+object. Keep in mind that checking the length of a generator BedTool will
+completely consume it.
+
+It's probably best to save intermediate versions to file using
+:meth:`BedTool.saveas`.  If you don't provide a filename, it'll save to an
+automatically cleaned up tempfile::
+
+    my_bedtool\
+     .filter(my_filter_func)\
+     .saveas()\
+     .intersect(y)\
+     .filter(lambda x: len(x) > 1000)\
+     .saveas('filtered-intersected-large.bed')
+
+
+"I'm getting a MalformedBedLineError"
+-------------------------------------
+This error can be raised by BEDTools itself.  Typical reasons are that start
+> end, or the fields are not tab-delimited.
+
+You can try the :func:`pybedtools.remove_invalid` function to clean up your
+file, or manually edit the offending lines.
+
+
+"I get a segfault when iterating over a BedTool object"
+-------------------------------------------------------
+
+`Issue #88 <https://github.com/daler/pybedtools/issues/88>`_ which
+addresses this issue -- in summary, Cython's handling of iterators works
+unexpectedly.  It's best to call the `next()` method explicitly when doing
+complex manipulations on an iterating :class:`BedTool`.
+
+
+"Can I add extra information to FASTA headers when using BedTool.sequence()?"
+-----------------------------------------------------------------------------
+
+Since BEDTools adds the feature name to the FASTA header, you can manipulate
+the feature name on the fly with a custom modifier function::
+
+    def fields2name(f):
+        "replace GFF featuretype field with the attributes field"
+        f[2] = f[-1]
+        return f
+
+    import pybedtools
+    g = pybedtools.BedTool("my.gff").each(fields2name).sequence(fi='my.fasta')
+
+    print open(g.seqfn).readline()
+
+
+"Too many files open" error
+---------------------------
+
+Sometimes you may get the error::
+
+    * Too many files open -- please submit a bug report so that this can be fixed
+
+This error occurs because you have hit your operating system's limit on the
+number of open files.  This usually happens when creating many :class:`BedTool`
+objects, often within a for-loop.
+
+In general, **try to create as few** :class:`BedTool` **objects as you can**.  Every time you
+create a :class:`BedTool` object, you create a new open file.  There is usually
+a BEDTools program that already does what you want, and will do it faster.
+
+
+For example, say we want to:
+
+* start with all annotations
+* only consider exons
+* write a file containing just exons
+* count reads in multiple BAM files for each exon
+
+
+Here is a first draft.  Note that the for-loop creates a :class:`BedTool`
+object each iteration, and the `result` is yet another :class:`BedTool`.  This
+will version will raise the "Too many files open" error.
+
+.. code-block:: python
+
+    # This version will be slow and, with many exons, will raise the "Too many
+    # files open" error
+
+    import pybedtools
+    all_features = pybedtools.BedTool('annotations.gff')
+    fout = open('exons.gff', 'w')
+    for feature in all_features:
+        if feature[2] != 'exon':
+            continue
+
+        fout.write(str(feature))
+
+        bt = pybedtools.BedTool([feature])
+        result = bt.multi_bam_coverage(bams=['reads1.bam', 'reads2.bam'])
+
+        # ...do something with result
+
+    fout.close()
+
+In contrast, it would be better to construct an "exon-only" :class:`BedTool` at
+the beginning.  The :meth:`BedTool.filter` method is a good way to do this.
+Then, there is only one call to :meth:`BedTool.multi_bam_coverage`.
+
+In this version there are only 3 :class:`BedTool` objects:  the
+one that opens `annotations.gff`, the one that uses `exons.gff` after it is
+saved, and `result`.  (Note that the one created from the filter operation is
+a "streaming" BedTool, so there is no open file that will contribute to the
+total).
+
+.. code-block:: python
+
+    # This is the recommended way.
+
+    import pybedtools
+
+    exons = pybedtools.BedTool('annotations.gff')\
+        .filter(lambda x: x[2] == 'exon')\
+        .saveas('exons.gff')
+
+    result = exons.multi_bam_coverage(bams=['reads1.bam', 'reads2.bam'])
+
+    # ...do something with result

pybedtools/source/docs/source/_static/custom.css

@@ -0,0 +1,3 @@
+div.highlight-python pre {
+	font-size: 0.7em;
+}

pybedtools/source/docs/source/_templates/layout.html

@@ -0,0 +1,6 @@
+{% extends '!layout.html' %}
+
+{% block relbar2 %}
+{{ relbar() }}
+
+{% endblock %}

pybedtools/source/docs/source/autodoc_source.rst

@@ -0,0 +1,360 @@
+
+.. _autodoc:
+
+.. _pybedtools reference:
+
+.. currentmodule:: pybedtools
+
+:mod:`pybedtools` Reference
+===========================
+The following tables summarize the methods and functions; click on a method or
+function name to see the complete documentation.
+
+.. contents::
+
+:class:`BedTool` creation
+-------------------------
+The main :class:`BedTool` documentation, with a list of all methods in
+alphabetical order at the bottom.  For more details, please see :ref:`creating
+a BedTool`.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool
+
+`BEDTools` wrappers
+-------------------
+These methods wrap `BEDTools` programs for easy use with Python; you can then
+use the other :mod:`pybedtools` functionality for further manipulation and
+analysis.
+
+The documentation of each of these methods starts with
+:mod:`pybedtools`-specific documentation, possibly followed by an example.
+Finally, the `BEDTools` help is copied verbatim from whatever version was
+installed when generating these docs.
+
+In general the `BEDTool` wrapper methods adhere to the :ref:`Design principles`:
+
+    * :ref:`temp principle`
+    * :ref:`similarity principle`
+    * :ref:`version principle`
+    * :ref:`default args principle`
+
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.intersect
+    pybedtools.bedtool.BedTool.window
+    pybedtools.bedtool.BedTool.closest
+    pybedtools.bedtool.BedTool.coverage
+    pybedtools.bedtool.BedTool.map
+    pybedtools.bedtool.BedTool.genome_coverage
+    pybedtools.bedtool.BedTool.merge
+    pybedtools.bedtool.BedTool.cluster
+    pybedtools.bedtool.BedTool.complement
+    pybedtools.bedtool.BedTool.subtract
+    pybedtools.bedtool.BedTool.slop
+    pybedtools.bedtool.BedTool.flank
+    pybedtools.bedtool.BedTool.sort
+    pybedtools.bedtool.BedTool.random
+    pybedtools.bedtool.BedTool.shuffle
+    pybedtools.bedtool.BedTool.annotate
+    pybedtools.bedtool.BedTool.multi_intersect
+    pybedtools.bedtool.BedTool.union_bedgraphs
+    pybedtools.bedtool.BedTool.pair_to_bed
+    pybedtools.bedtool.BedTool.pair_to_pair
+    pybedtools.bedtool.BedTool.bam_to_bed
+    pybedtools.bedtool.BedTool.to_bam
+    pybedtools.bedtool.BedTool.bedpe_to_bam
+    pybedtools.bedtool.BedTool.bed6
+    pybedtools.bedtool.BedTool.bam_to_fastq
+    pybedtools.bedtool.BedTool.sequence
+    pybedtools.bedtool.BedTool.mask_fasta
+    pybedtools.bedtool.BedTool.nucleotide_content
+    pybedtools.bedtool.BedTool.multi_bam_coverage
+    pybedtools.bedtool.BedTool.tag_bam
+    pybedtools.bedtool.BedTool.jaccard
+    pybedtools.bedtool.BedTool.reldist
+    pybedtools.bedtool.BedTool.overlap
+    pybedtools.bedtool.BedTool.links
+    pybedtools.bedtool.BedTool.igv
+    pybedtools.bedtool.BedTool.window_maker
+    pybedtools.bedtool.BedTool.groupby
+    pybedtools.bedtool.BedTool.expand
+
+Other :class:`BedTool` methods
+------------------------------
+These methods are some of the ways in which :mod:`pybedtools` extend the
+BEDTools suite.
+
+
+Feature-by-feature operations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Methods that operate on a feature-by-feature basis to modify or filter features
+on the fly.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.each
+    pybedtools.bedtool.BedTool.filter
+    pybedtools.bedtool.BedTool.split
+    pybedtools.bedtool.BedTool.truncate_to_chrom
+    pybedtools.bedtool.BedTool.remove_invalid
+
+The :mod:`pybedtools.featurefuncs` module contains some commonly-used functions
+that can be passed to :meth:`BedTool.each`:
+
+.. currentmodule:: pybedtools
+
+.. autosummary::
+    :toctree:
+
+    pybedtools.featurefuncs.three_prime
+    pybedtools.featurefuncs.five_prime
+    pybedtools.featurefuncs.TSS
+    pybedtools.featurefuncs.extend_fields
+    pybedtools.featurefuncs.center
+    pybedtools.featurefuncs.midpoint
+    pybedtools.featurefuncs.normalized_to_length
+    pybedtools.featurefuncs.rename
+    pybedtools.featurefuncs.greater_than
+    pybedtools.featurefuncs.less_than
+    pybedtools.featurefuncs.normalized_to_length
+    pybedtools.featurefuncs.rename
+    pybedtools.featurefuncs.bedgraph_scale
+    pybedtools.featurefuncs.add_color
+    pybedtools.featurefuncs.gff2bed
+    pybedtools.featurefuncs.bed2gff
+
+
+
+Searching for features
+~~~~~~~~~~~~~~~~~~~~~~
+These methods take a single interval as input and return the intervals of the
+BedTool that overlap.
+
+This can be useful when searching across many BED files for a particular
+coordinate range -- for example, they can be used identify all binding sites,
+stored in many different BED files, that fall within a gene's coordinates.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.all_hits
+    pybedtools.bedtool.BedTool.any_hits
+    pybedtools.bedtool.BedTool.count_hits
+    pybedtools.bedtool.BedTool.tabix_intervals
+    pybedtools.bedtool.BedTool.tabix
+    pybedtools.bedtool.BedTool.bgzip
+
+
+:class:`BedTool` introspection
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+These methods provide information on the :class:`BedTool` object.
+
+If using :meth:`BedTool.head`, don't forget that you can index into
+:class:`BedTool` objects, too.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.head
+    pybedtools.bedtool.BedTool.count
+    pybedtools.bedtool.BedTool.field_count
+    pybedtools.bedtool.BedTool.file_type
+
+
+Randomization helpers
+~~~~~~~~~~~~~~~~~~~~~
+Helper methods useful for assessing empirical instersection
+distributions between interval files.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.parallel_apply
+    pybedtools.bedtool.BedTool.randomstats
+    pybedtools.bedtool.BedTool.randomintersection
+    pybedtools.bedtool.BedTool.randomintersection_bp
+    pybedtools.bedtool.BedTool.random_subset
+    pybedtools.bedtool.BedTool.random_jaccard
+    pybedtools.bedtool.BedTool.random_op
+
+Managing :class:`BedTool` objects on disk
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+These methods are used to specify where to save results from :class:`BedTool`
+operations.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.saveas
+    pybedtools.bedtool.BedTool.moveto
+
+
+Misc operations
+~~~~~~~~~~~~~~~
+Methods that can't quite be categorized into the above sections.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.BedTool.cat
+    pybedtools.bedtool.BedTool.at
+    pybedtools.bedtool.BedTool.absolute_distance
+    pybedtools.bedtool.BedTool.cut
+    pybedtools.bedtool.BedTool.total_coverage
+    pybedtools.bedtool.BedTool.with_attrs
+    pybedtools.bedtool.BedTool.as_intervalfile
+    pybedtools.bedtool.BedTool.introns
+    pybedtools.bedtool.BedTool.set_chromsizes
+    pybedtools.bedtool.BedTool.print_sequence
+    pybedtools.bedtool.BedTool.save_seqs
+    pybedtools.bedtool.BedTool.seq
+    pybedtools.bedtool.BedTool.liftover
+    pybedtools.bedtool.BedTool.colormap_normalize
+    pybedtools.bedtool.BedTool.relative_distance
+
+Module-level functions
+----------------------
+
+Working with example files
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+:mod:`pybedtools` comes with many example files.  Here are some useful
+functions for accessing them.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.bedtool.example_bedtool
+    pybedtools.filenames.list_example_files
+    pybedtools.filenames.example_filename
+
+Creating :class:`Interval` objects from scratch
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+:class:`Interval` objects are the core object in :mod:`pybedtools` to represent
+a genomic interval, written in Cython for speed.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.cbedtools.Interval
+    pybedtools.cbedtools.create_interval_from_list
+
+:mod:`pybedtools` setup and config
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Use these functions right after importing in order to use custom paths or to
+clean up the temp directory.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.helpers.set_bedtools_path
+    pybedtools.helpers.get_tempdir
+    pybedtools.helpers.set_tempdir
+    pybedtools.helpers.cleanup
+    pybedtools.debug_mode
+
+
+Working with "chromsizes" or assembly coordinate files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Many `BEDTools` programs need "genome files" or "chromsizes" files so as to
+remain within the coordinates of the assembly you're working on.  These
+functions help manage these files.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.helpers.get_chromsizes_from_ucsc
+    pybedtools.helpers.chromsizes
+    pybedtools.helpers.chromsizes_to_file
+
+
+Performing operations in parallel (multiprocessing)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.parallel.parallel_apply
+
+:mod:`pybedtools.contrib`
+-------------------------
+The :mod:`pybedtools.contrib` module contains higher-level code that leverages
+:class:`BedTool` objects for common analyses.
+
+
+Plotting
+~~~~~~~~
+Plotting results from BEDTools/pybedtools operations is very useful for
+exploring and understanding the tools as well as for teaching purposes.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.plotting.Track
+    pybedtools.contrib.plotting.TrackCollection
+    pybedtools.contrib.plotting.binary_heatmap
+    pybedtools.contrib.plotting.binary_summary
+    pybedtools.contrib.plotting.BedToolsDemo
+    pybedtools.contrib.plotting.ConfiguredBedToolsDemo
+
+
+
+
+Working with bigWig files
+~~~~~~~~~~~~~~~~~~~~~~~~~
+At this time, :mod:`pybedtools` does not support reading bigWig files, only
+creating them via UCSC utilities.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.bigwig.bam_to_bigwig
+    pybedtools.contrib.bigwig.bedgraph_to_bigwig
+    pybedtools.contrib.bigwig.wig_to_bigwig
+
+Working with bigBed files
+~~~~~~~~~~~~~~~~~~~~~~~~~
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.bigbed.bigbed
+    pybedtools.contrib.bigbed.bigbed_to_bed
+
+
+:class:`IntersectionMatrix`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The :class:`IntersectionMatrix` class makes it easy to intersect a large number
+of interval files with each other.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.IntersectionMatrix
+
+:mod:`contrib.venn_maker`
+~~~~~~~~~~~~~~~~~~~~~~~~~
+The :mod:`venn_maker` module helps you make Venn diagrams using the R package
+`VennDiagram <http://www.biomedcentral.com/1471-2105/12/35>`_.
+
+Note that Venn diagrams are not good for when you have nested intersections.
+See the docs for :func:`pybedtools.contrib.venn_maker.cleaned_intersect` and
+its source for more details.
+
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.venn_maker
+    pybedtools.contrib.venn_maker.venn_maker
+    pybedtools.contrib.venn_maker.cleaned_intersect
+
+:mod:`contrib.long_range_interaction`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. autosummary::
+    :toctree: autodocs
+
+    pybedtools.contrib.long_range_interaction.tag_bedpe
+    pybedtools.contrib.long_range_interaction.cis_trans_interactions

pybedtools/source/docs/source/autodocs/pybedtools.contrib.plotting.Track.rst

@@ -0,0 +1,166 @@
+pybedtools.contrib.plotting.Track
+=================================
+
+.. currentmodule:: pybedtools.contrib.plotting
+
+.. autoclass:: Track
+
+   
+   .. automethod:: __init__
+
+   
+   .. rubric:: Methods
+
+   .. autosummary::
+   
+      ~Track.__init__
+      ~Track.add_callback
+      ~Track.add_checker
+      ~Track.autoscale
+      ~Track.autoscale_None
+      ~Track.changed
+      ~Track.check_update
+      ~Track.contains
+      ~Track.convert_xunits
+      ~Track.convert_yunits
+      ~Track.draw
+      ~Track.findobj
+      ~Track.format_cursor_data
+      ~Track.get_agg_filter
+      ~Track.get_alpha
+      ~Track.get_animated
+      ~Track.get_array
+      ~Track.get_capstyle
+      ~Track.get_children
+      ~Track.get_clim
+      ~Track.get_clip_box
+      ~Track.get_clip_on
+      ~Track.get_clip_path
+      ~Track.get_cmap
+      ~Track.get_contains
+      ~Track.get_cursor_data
+      ~Track.get_dashes
+      ~Track.get_datalim
+      ~Track.get_ec
+      ~Track.get_edgecolor
+      ~Track.get_edgecolors
+      ~Track.get_facecolor
+      ~Track.get_facecolors
+      ~Track.get_fc
+      ~Track.get_figure
+      ~Track.get_fill
+      ~Track.get_gid
+      ~Track.get_hatch
+      ~Track.get_in_layout
+      ~Track.get_joinstyle
+      ~Track.get_label
+      ~Track.get_linestyle
+      ~Track.get_linestyles
+      ~Track.get_linewidth
+      ~Track.get_linewidths
+      ~Track.get_ls
+      ~Track.get_lw
+      ~Track.get_offset_position
+      ~Track.get_offset_transform
+      ~Track.get_offsets
+      ~Track.get_path_effects
+      ~Track.get_paths
+      ~Track.get_picker
+      ~Track.get_pickradius
+      ~Track.get_rasterized
+      ~Track.get_sizes
+      ~Track.get_sketch_params
+      ~Track.get_snap
+      ~Track.get_tightbbox
+      ~Track.get_transform
+      ~Track.get_transformed_clip_path_and_affine
+      ~Track.get_transforms
+      ~Track.get_url
+      ~Track.get_urls
+      ~Track.get_visible
+      ~Track.get_window_extent
+      ~Track.get_xlims
+      ~Track.get_zorder
+      ~Track.have_units
+      ~Track.is_transform_set
+      ~Track.pchanged
+      ~Track.pick
+      ~Track.pickable
+      ~Track.properties
+      ~Track.remove
+      ~Track.remove_callback
+      ~Track.set
+      ~Track.set_aa
+      ~Track.set_agg_filter
+      ~Track.set_alpha
+      ~Track.set_animated
+      ~Track.set_antialiased
+      ~Track.set_antialiaseds
+      ~Track.set_array
+      ~Track.set_capstyle
+      ~Track.set_clim
+      ~Track.set_clip_box
+      ~Track.set_clip_on
+      ~Track.set_clip_path
+      ~Track.set_cmap
+      ~Track.set_color
+      ~Track.set_contains
+      ~Track.set_dashes
+      ~Track.set_ec
+      ~Track.set_edgecolor
+      ~Track.set_edgecolors
+      ~Track.set_facecolor
+      ~Track.set_facecolors
+      ~Track.set_fc
+      ~Track.set_figure
+      ~Track.set_gid
+      ~Track.set_hatch
+      ~Track.set_in_layout
+      ~Track.set_joinstyle
+      ~Track.set_label
+      ~Track.set_linestyle
+      ~Track.set_linestyles
+      ~Track.set_linewidth
+      ~Track.set_linewidths
+      ~Track.set_ls
+      ~Track.set_lw
+      ~Track.set_norm
+      ~Track.set_offset_position
+      ~Track.set_offsets
+      ~Track.set_path_effects
+      ~Track.set_paths
+      ~Track.set_picker
+      ~Track.set_pickradius
+      ~Track.set_rasterized
+      ~Track.set_sizes
+      ~Track.set_sketch_params
+      ~Track.set_snap
+      ~Track.set_transform
+      ~Track.set_url
+      ~Track.set_urls
+      ~Track.set_verts
+      ~Track.set_verts_and_codes
+      ~Track.set_visible
+      ~Track.set_zorder
+      ~Track.to_rgba
+      ~Track.update
+      ~Track.update_from
+      ~Track.update_scalarmappable
+   
+   
+
+   
+   
+   .. rubric:: Attributes
+
+   .. autosummary::
+   
+      ~Track.axes
+      ~Track.midpoint
+      ~Track.mouseover
+      ~Track.stale
+      ~Track.sticky_edges
+      ~Track.update_dict
+      ~Track.zorder
+   
+   

pybedtools/source/docs/source/changes.rst

@@ -0,0 +1,880 @@
+.. include:: includeme.rst
+
+Changelog
+=========
+
+Changes in v0.12.0
+------------------
+
+2025-03-14
+
+* Allow ``pathlib.Path`` objects to be used in arbitrary ``BedTool``  methods.
+  Previously, only ``BedTool`` creation supported ``Path`` (fixes #421)
+* Add support for Python 3.13, thanks to @theAeon Andrew Robbins (fixes #422)
+
+
+Changes in v0.11.0
+------------------
+
+2025-01-02
+
+* Add type hints and clean up code in general (big thanks to @duartemolha)
+* Fix post-installation tests in Debian package (thanks @mr-c)
+* Remove Python 3.8 support which reached end-of-life 2024-10-07.
+
+Changes in v0.10.1
+------------------
+
+2024-04-09
+
+* Remove last traces of Python 2.7 support by removing ``six`` dependency (thanks Valentyn Bezshapkin)
+* Support building on later C++ toolchains (thanks Cameron Smith)
+* Support ``pathlib.Path`` in ``BedTool.cat()`` (fixes #405)
+* Improvements to testing: add tests for Python 3.12, more explicit setup/teardown
+
+Changes in v0.9.1
+-----------------
+
+2023-07-23
+
+* Dropping support for Python 3.6 and 3.7
+* Respect sorting of chromsize files (thanks @mgperry)
+* Updated setup.py to correctly reflect the MIT license change elsewhere (`#374
+  <https://github.com/daler/pybedtools/issues/374>`, thanks @hyandell)
+*  Support plotting lengths of intervals and custom DPI (`#367
+   <https://github.com/daler/pybedtools/issues/367>`, `#366
+   <https://github.com/daler/pybedtools/issues/366>`), thanks @yunfeiguo)
+* Remove outdated hard-coded check for 510 files in ``intersect`` and instead
+  defer to local machine's ``ulimit``
+* Enabling building/installing on Python 3.11 (thanks @daz10000)
+* Allow np.int64 start/stop positions to be used when creating Interval objects (`#390 <https://github.com/daler/pybedtools/issues/390>`)
+* properly close filehandles in .save_seq (thanks @PeterRobots)
+* include minimal pyproject.toml file (thanks @afg1)
+
+
+Changes in v0.9
+---------------
+
+2022-01-23
+
+The biggest change is that pybedtools is now under the MIT license, following
+the lead of BEDTools itself.
+
+Bugfixes:
+
+* Bugfix: `Interval` objects representing VCF lines now have their `start`
+  attribute correctly zero-based, as indicated in the docs and consistent with
+  other 1-based formats. See `#355 <https://github.com/daler/pybedtools/issues/355>`_.
+* Bugfix: Manually creating `Interval` objects using the `otherfields` argument
+  now correctly converts to C++ strings. See `#348
+  <https://github.com/daler/pybedtools/issues/348>`_.
+* Bugfix: Workaround for `BedTool.intersect` which in some versions of BEDTools
+  requires a specific order of arguments. Fixes `#345
+  <https://github.com/daler/pybedtools/issues/345>`_ and also is a better way
+  of addressing `#81 <https://github.com/daler/pybedtools/issues/81>`_.
+
+Code cleanup:
+
+* Removed some remnants of Python 2.7 support (thanks @DavidCain)
+* Updates to setup.py classifiers to better reflect state of code (thanks @DavidCain)
+* Sorted filenames in setup.py to improve reproducibility of build (thanks @lamby)
+* Tests converted to run on GitHub Actions (see `#339
+  <https://github.com/daler/pybedtools/pull/339>`_).
+
+Changes in v0.8.2
+-----------------
+
+2021-03-13
+
+Minor updates
+
+* Removed scripts directory from installed version. These are still available
+  in the GitHub repo, but were causing import issues with Python 3.9 and were
+  not well-used in the first place.
+* Bugfix: unicode is better handled in gzipped files (thanks @naumenko-sa, see
+  `#320 <https://github.com/daler/pybedtools/pull/320>`_)
+* Bugfix: correctly ignore warnings even with capital letters (thanks
+  @JureZmrzlikar, see `#326 <https://github.com/daler/pybedtools/pull/326>`_)
+* Bugfix/improvements: update tests and code to work with Python 3.8 (see `#324
+  <https://github.com/daler/pybedtools/pull/324>`_). Also addresses `#322
+  <https://github.com/daler/pybedtools/issues/322>`_.
+* Improvement: updates tests to work with bedtools v2.30
+* Improvement: integration of `genomepy
+  <https://github.com/vanheeringen-lab/genomepy>`_, which if installed will
+  help retrieve chromsizes files for less commonly used assemblies (thanks
+  @simonvh, see `#323 <https://github.com/daler/pybedtools/pull/323/>`_)
+
+
+Changes in v0.8.1
+-----------------
+
+2019-12-27
+
+This version has minor updates and bugfixes:
+
+* Bugfix: Fixes to `pbt_plotting_example.py` (thanks Steffen Möllera @smoe)
+* Bugfix: Using `BedTool.saveas()` when a BedTool already points to a compressed file
+  correctly saves (`#308 <https://github.com/daler/pybedtools/pull/308>`_)
+* Improvement: Deprecate `pandas.read_table` (thanks André F. Rendeiro
+  @afrendeiro)
+* Improvement: overhaul testing on travis-ci
+* Improvement: BedTool objects support either strings or Path objects (`#287
+  <https://github.com/daler/pybedtools/pull/287>`_, thanks @drchsl)
+* Improvement: MySQL host can be configured (`#301
+  <https://github.com/daler/pybedtools/pull/301>`_, thanks André F. Rendeiro
+  @afrendeiro)
+* Improvement: Better version string parsing (`#289
+  <https://github.com/daler/pybedtools/pull/289>`_, thanks Steffen Möllera
+  @smoe), fixes `#275 <https://github.com/daler/pybedtools/issues/275>`_ and others.
+* Improvement: Proper CRAM support: `#307 <https://github.com/daler/pybedtools/pull/307>`_
+* Improvement: Raise an exception when the `-b` argument to `BedTool.intersect` has more
+  than 510 files (`#303 <https://github.com/daler/pybedtools/pull/306>`_)
+* `*.h` files now included in the distribution (thanks @blaiseli)
+* Improvement: Update tests to work with bedtools v2.29.2
+
+
+Changes in v0.8.0
+-----------------
+
+2018-11-28
+
+This version further improves testing, improves the way C++ files are included
+in the package, and fixes many long-standing bugs.
+
+* Using pytest framework rather than nose for testing
+* Updated `setup.py` to be more robust and to more clearly separate
+  "cythonization" into .cpp files
+* Updated test harness for testing in independent conda environments
+* All issue tests go in their own test module
+* Included Python 3.7 tests (note that at the time of this writing, pysam is
+  not yet available on bioconda so that dependency is pip-installed in the
+  test) (`#254 <https://github.com/daler/pybedtools/issues/254>`_)
+* Updated tests to reflect BEDTool 2.27.1 output (`#260
+  <https://github.com/daler/pybedtools/issues/260>`_`#261
+  <https://github.com/daler/pybedtools/issues/261>`_)
+* Removed the `contrib.classifier` module, which has been unsupported for
+  a while.
+* More informative error messages for UCSC tools if they're missing (`#227
+  <https://github.com/daler/pybedtools/issues/227>`_)
+* BedTool objects that are the result of operations that create files that are
+  not BED/GTF/GFF/BAM can be more easily converted to pandas.DataFrame with
+  `disable_auto_names=True` arg to `BedTool.to_dataframe()` (`#258
+  <https://github.com/daler/pybedtools/issues/258>`_)
+* Added aliases to existing methods to match current BEDTools commands, e.g.
+  the `BedTool.nucleotide_content` method can now also be called using
+  `BedTool.nuc` which is consistent with the `bedtools nuc` command line name.
+* New wrapper for `bedtools split`. The wrapper method is called `splitbed` to
+  maintain backwards compatibility because `pybedtools.BedTool` objects have
+  long had a `split` method that splits intervals based on a custom function.
+* New wrapper for `bedtools spacing`.
+* `BedTool.from_dataframe` handles NaN in dataframes by replacing with `"."`,
+  and is more explicit about kwargs that are passed to `pandas.DataFrame`
+  (`#257 <https://github.com/daler/pybedtools/issues/257>`_)
+* Raise FileNotFoundError when on Python 3 (thanks Gosuke Shibahara, (`#255
+  <https://github.com/daler/pybedtools/issues/255>`_)
+* Relocated BEDTools header and .cpp files to the `pybedtools/include`
+  directory, so they can more easily be linked to from external packages
+  (`#253 <https://github.com/daler/pybedtools/issues/253>`_)
+* Add test for (`#118 <https://github.com/daler/pybedtools/issues/118>`_)
+* `BedTool.tabix_contigs` will list the sequence names indexed by tabix
+  (`#180 <https://github.com/daler/pybedtools/issues/180>`_)
+* `BedTool.tabix_intervals` will return an empty generator if the coordinates
+  provided are not indexed, unless `check_coordinates=True` in which case the
+  previous behavior of raising a ValueError is triggered (`#181
+  <https://github.com/daler/pybedtools/issues/181>`_)
+* Bugfix: Avoid "ResourceWarning: unclosed file" in `helpers.isBGZIP` (thanks
+  Stephen Bush)
+* Bugfix: Interval objects created directly no longer have their filetype set
+  to None (`#217 <https://github.com/daler/pybedtools/issues/217>`_)
+* Bugfix: Fixed the ability to set paths and reload module afterwards (`#218
+  <https://github.com/daler/pybedtools/issues/218>`_, `#220
+  <https://github.com/daler/pybedtools/issues/220>`_, `#222
+  <https://github.com/daler/pybedtools/issues/222>`_)
+* Bugfix: `BedTool.head()` no longer uses an IntervalIterator (which would
+  check to make sure lines are valid BED/GTF/GFF/BAM/SAM). Instead, it simply
+  prints the first lines of the underlying file.
+* Bugfix: functions passed to `BedTool.filter` and `BedTool.each` no longer
+  silently pass ValueErrors (`#231
+  <https://github.com/daler/pybedtools/issues/231>`_)
+* Bugfix: Fixed IndexError in IntervalIterator if there was an empty line (`#233
+  <https://github.com/daler/pybedtools/issues/233>`_)
+* Bugfix: Add additional constraint to SAM file detection to avoid incorrectly
+  detecting a BED file as SAM (`#246
+  <https://github.com/daler/pybedtools/issues/246>`_)
+* Bugfix: accessing Interval.fields after accessing Interval.attrs no longer
+  raises ValueError (`#246 <https://github.com/daler/pybedtools/issues/246>`_)
+
+Changes in v0.7.10
+------------------
+
+2017-05-31
+
+Various bug fixes and some minor feature additions:
+
+* Support for comma-separated lists for `mapBed` (thanks Chuan-Sheng Foo)
+* Support many calls to `tabix_intervals` without hitting a "Too many open
+  files" error (`#190 <https://github.com/daler/pybedtools/issues/190>`_)
+* Clarify documentation for `genome_coverage` when used with default
+  parameters (`#113 <https://github.com/daler/pybedtools/issues/113>`_)
+* Ignore stderr from samtools on older zlib versions (`#209 <https://github.com/daler/pybedtools/issues/209>`_, thanks Gert Hulselmans)
+* Support fetching all regions from a chromosome (`#201 <https://github.com/daler/pybedtools/issues/210>`_, thanks Matt Stone)
+* Add wrapper for `shiftBed` (`#200 <https://github.com/daler/pybedtools/issues/200>`_, thanks Saket Choudhary)
+* Fix `truncate_to_chrom` in Python 3 (`#203 <https://github.com/daler/pybedtools/issues/203>`_, thanks Saulius Lukauskas)
+* When making bigWigs, use `bedSort` to ensure the sort order matches that expected by UCSC tools (`#178 <https://github.com/daler/pybedtools/issues/178>`_)
+* Fix newline handling of `pysam.ctabix.TabixIterator` output (`#196 <https://github.com/daler/pybedtools/issues/196>`_)
+
+
+Changes in v0.7.9
+-----------------
+
+2017-01-25
+
+Minor bugfix release:
+
+* add `contrib.bigwig.bigwigtobedgraph` (thanks Justin Fear)
+* fix `BedTool.seq()` in Python 3
+* fix intron creation (`#182 <https://github.com/daler/pybedtools/pull/182>`_, thanks @mmendez12)
+* add `six` as an explicit requirement (`#184 <https://github.com/daler/pybedtools/pull/184>`_, thanks @jrdemasi)
+* improvements to setup (`<https://github.com/daler/pybedtools/issues/185>`_)
+* make pandas fully optional
+
+Changes in v0.7.8
+-----------------
+
+2016-07-13
+
+* Be more careful about BAM vs bgzipped files (#168)
+* `BedTool.bgzip` now preserves the header when sorting
+* In Python 3, parsed BEDTools help string is decoded properly
+* Ensure integer number of processes in Python 3 (thanks Illa Shamovsky)
+* Add details on IOError messages for broken pipe error
+* Make converting to pandas.DataFrames easier with non-standard BED files (thanks Panos Firmpas)
+
+Changes in v0.7.7
+-----------------
+
+2016-03-11
+
+* Chromsizes for dm6 and mm10 assemblies added to `genome_registry`
+* Better Python 3 compatibility in the `long_range_interaction` module
+* New `featurefuncs.UniqueID` class, useful for ensuring all features in a file
+  have a unique ID in their name field.
+* Fix error message when a specified genome file doesn't exist (thanks Saket Choudhary)
+
+Changes in v0.7.6
+-----------------
+
+2016-02-01
+
+* New module `pybedtools.contrib.long_range_interaction` for working with
+  HiC-like data.
+
+Changes in v0.7.5
+-----------------
+
+2016-01-25
+
+* When using tabix-indexed files, `tabix` and `bgzip` are no longer required to
+  be installed separately. Only `pysam` is needed.
+
+* Recent BEDTools releases support multiple files for the `-b` argument of
+  `bedtools intersect`. This version of `pybedtools` now supports multiple
+  files as well. Note that it is still possible to provide a list of strings
+  representing intervals as the `b` argument to `BedTool.intersect`. To
+  differentiate between a list of intervals and a list of filenames, the first
+  item converted into an `Interval` object; if it fails then consider the items
+  to be filenames; otherwise assume strings representing intervals. This check
+  only occurs if the `b` argument is a list or tuple; other iterable types are
+  always assumed to be intervals.
+
+Changes in v0.7.4
+-----------------
+
+2015-10-30
+
+Bugfix release.
+
+- fix `#147 <https://github.com/daler/pybedtools/issues/147>`_ so that warnings
+  are simply passed to the user without raising exceptions
+- in setup.py, allow depedencies to have "loose" versions with suffixes like
+  "rc1"
+- fix in `BedTool.cat()` on empty files (thanks Brad Chapman (`PR #149
+  <https://github.com/daler/pybedtools/pull/149>`_)
+
+Changes in v0.7.1
+-----------------
+
+2015-09-28
+
+This is largely a bugfix release with the following changes:
+
+- fix for some BAM headers (thanks Gabriel Platt)
+- unified IntervalIterator to address some streaming issues (fixes #143)
+- fix bug where `__add__` was not re-raising exceptions (thanks Brad Chapman
+  and Dan Halligan)
+
+
+Changes in v0.7.0
+-----------------
+
+2015-09-21
+
+This release reflects a major upgrade in the underlying code in order to
+support both Python 2 and Python 3 using the same code. Aside from trivial
+things like converting print statements to functions and using `next()` instead
+of `.next()`, this required a substantial rewrite to support the way strings
+are handled in Python 3 (in Cython and wrapped C++) and how relative modules
+work.
+
+Importantly, after converting them to Python 2- and 3-compatible syntax *all
+previous tests pass* so to the end user should not notice any differences
+except those noted below.
+
+Strings from Interval fields are unicode
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+For consistency between Python 2 and 3, all strings from Interval objects are
+now unicode. That is, in Python 2, previously we would get this::
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> a[0].name
+    'feature1'
+
+Now, we get this::
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> a[0].name
+    u'feature1'
+
+
+samtools no longer a dependency
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The dependency for samtools has been removed, which simplifies the installation
+process. Instead, `pysam` is used for handling BAM files.
+
+In order for existing tests to pass, `pysam.AlignedSegment` objects are
+currently converted to `pybedtools.Interval` objects when iterating over a BAM
+file. This will come at a performance cost if you are iterating over all reads
+in a BAM file using the `pybedtools.BAM` object.
+
+In the future, iterating over a BAM file will yield `pysam.AlignedSegment`
+objects directly, but for now you can use the `pybedtools.BAM.pysam_bamfile`
+attribute to access the underlying `pysam.AlignmentFile`
+
+Cython no longer a dependency
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The Cythonized ``.cxx`` files are now shipped with the `pybedtools`
+distribution, so Cython is no longer a requirement for installation.
+
+You will however need to have Cython installed if you're developing pybedtools.
+
+Remote BAM support clarification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Previously, `pybedtools` was able to support remote BAMs by loosely wrapping
+samtools, but BAM files still needed to be fully downloaded to disk before
+using with BEDTools. This was done automatically, but through an inefficient
+mechanism.
+
+Pysam does support remote BAMs, and as before, a BAM file needs to be created
+on disk for use with BEDTools. But now this needs to be explicitly done by the
+user, which should result in better performance.
+
+
+Iterating over intervals
+~~~~~~~~~~~~~~~~~~~~~~~~
+Previously, when iterating over a `BedTool` object, different machinery would
+be invoked depending on whether the BedTool was pointing to a file (a
+cbedtools.IntervalFile would be invoked), to another iterator of Interval
+objects, or to a stream like from the stdout of a BEDTools call
+(cbedtools.IntervalIterator in both cases).
+
+Everything is now an IntervalIterator, simplifying the path towards
+performance optimization.
+
+gzip support
+~~~~~~~~~~~~
+Thanks to Saulius Lukauskas, gzip handling is now improved, and calling
+`BedTool.saveas()` with a `.gz` extension on the filename will automatically
+compress the output.
+
+Docker
+~~~~~~
+In the github repo there is a `docker` directory containing Dockerfiles to set
+up isolated testing environments. These Dockerfiles also demonstrate how to set
+up a complete environment starting from a base Ubuntu install.
+
+Tests
+~~~~~
+All tests from v0.6.9 (which was Python 2 only) have been made Python 2/3
+compatible and all previous tests pass.
+
+If you have docker installed, from the top level directory, you can run the
+full tests like this::
+
+    cd docker
+    ./full-tests.sh
+
+This will build docker containers for Python 2 and Python 3 with all
+depedencies, export the parent directory to the container, and run the test
+suite.
+
+
+Conda packages
+~~~~~~~~~~~~~~
+You can now install the latest versions of tabix, bedtools, pysam, and
+pybedtools from conda, dramatically speeding up installation time. These
+mechanisms are used for automated testing as well (see the ``condatest.sh``
+script in the github repo).
+
+To use these packages in your own environment(s), specify the `daler` conda
+channel like this::
+
+    conda install -c daler pybedtools
+
+Note that this will not install BEDTools or tabix unless you explicitly say
+so::
+
+    conda install -c daler pybedtools bedtools tabix
+
+.. note::
+
+    This currently only works on Linux; contributions to Mac conda recipes (see
+    the `conda` dir in the github repo) would be welcomed.
+
+Changes in v0.6.9
+-----------------
+
+2014-12-11
+
+Minor bug fix release.
+
+* improved the automatic field name handling when converting an interval file to
+  a `pandas DataFrame`.
+* fixed a bug in `IntervalFile` methods `all_hits`, `any_hits` and `count_hits`
+  where zero-length features were being counted multiple times (thanks Brent
+  Pedersen and Kyle Smith)
+* bgzip and tabix paths can now be configured separately (thanks Rob Beagrie)
+* fixed a bug where streaming BAM files were read fully into memory (thanks
+  Alexey Sergushichev)
+
+Changes in v0.6.8
+-----------------
+
+2014-10-08
+
+Bugfix: Thanks to Gabriel Pratt, `pybedtools` is no longer plagued by open filehandles
+in the C code causing the notorious "Too many files open" error.
+
+Changes in v0.6.7
+-----------------
+
+2014-06-01
+
+Now compatible with BEDTools v2.21.0.
+
+The one exception is that the new `bedtools intersect` functionality that
+allows multiple `-b` files is not yet implemented in `pybedtools`.
+
+New features:
+
+* `BedTool.fisher()` wraps the new BEDTools `fisher` tool.  The result is
+  an object containing parsed results.
+
+* `BedTool.colormap_normalize()` accepts a `percentile` argument, useful when
+  applying colormaps to data with a handful of extreme outliers
+
+* `BedTool.to_datafame()` converts a `BedTool` object into a `pandas.DataFrame`
+  with columns named after the appropriate fields for the filetype (thanks
+  Radhouane Aniba for the suggestion)
+
+* `BedTool.tail()` to complement `BedTool.head()` (thanks Radhouane Aniba for
+  the suggestion)
+
+* Add hg38 and hg38.default chromsizes
+
+Minor bug fixes:
+
+* Ensure tuple-like args to `parallel_apply` (fixes #109)
+
+* Temp fix for BEDTools v2.20.0 which required the `-w` arg to come before the
+  `-s` arg in `bedtools makewindows` (#81)
+
+* Better (i.e., UCSC Genome Browser-compliant) defaults for `featurefuncs.expand_fields`.
+
+* Fix for BedTool.all_hits() and any_hits() which will now show hits for
+  zero-length features intersecting with other zero-length features with the
+  same coordinates.
+
+
+
+Changes in v0.6.6
+-----------------
+
+2014-05-23
+
+This is a compatibility release, updated for BEDTools v2.20.0.
+
+There is one API change that affects the behavior of overloaded operators (that
+is, using `+` and `-` with BedTool objects) when one of the BedTool objects
+represents an empty file.
+
+Assume `a` is a BedTool object representing a regular BED file but `b` is
+empty.  Previously:
+
+    * a + b = a
+    * b + a = b
+    * a - b = a
+    * b - a = a
+    * b - b = b
+    * a + a = a
+
+The following changes have been made (indicated in **bold**), which hopefully
+make more logical sense:
+
+    * **a + b = b**
+    * b + a = b
+    * a - b = a
+    * **b - a = b**
+    * b - b = b
+    * a + a = a
+
+Changes in v0.6.5
+-----------------
+
+2014-04-24
+
+This is a minor bug-fix release:
+
+* Fix for BedTool.all_hits() and any_hits() which will now show hits for
+  zero-length features with the same coordinates, like the rest of BEDTools.
+
+* Improved error-handling to avoid Python interpreter crashing in cases when
+  a BED file on the filesystem becomes unavailable after a BedTool object has
+  been created for it.
+
+
+Changes in v0.6.4
+-----------------
+
+2014-01-08
+
+* Full integration with BEDTools v2.18.  This includes some compatibility fixes
+  for the new buffered output capabilities of BEDTool `intersect` and wrapping
+  the new `bedtools sample` tool.
+
+* Overloaded operators (`+` and `-`) allow empty files as input, even using
+  BEDTools v2.18+.
+
+* Travis-CI builds now use BEDTools v2.18+ for tests.
+
+* Fix for :func:`pybedtools.featurefuncs.midpoint` (thanks ny-shao)
+
+* Fix to :meth:`BedTool.randomstats` (thanks Michael Reschen)
+
+
+Changes in v0.6.3
+-----------------
+
+2013-12-16
+
+* New :mod:`pybedtools.parallel` module for working with many operations in
+  parallel.  See the docs for :func:`pybedtools.parallel.parallel_apply` for
+  details.
+
+* :func:`pybedtools.contrib.bigbed.bigbed` for converting to bigBed format,
+  along with auto-SQL creation as needed.
+
+* New function :func:`pybedtools.contrib.bigbed.bigbed_to_bed`, so now bigBed
+  -> BED and BED -> bigBed interconversions are trivial.
+
+* Support for remote BAMs by passing `remote=True` when creating
+  a :class:`BedTool` object
+
+* New method :meth:`BedTool.at` for subsetting a BedTool by a set of (sorted)
+  indexes.
+
+* New functions :func:`featurefuncs.gff2bed` and :func:`featurefuncs.bed2gff`
+  for use with :meth:`BedTool.each`, for easy converting GFF/GTF to BED
+
+* New function :func:`add_color` for applying matplotlib colormaps to BED
+  files; see also new method :meth:`pybedtools.BedTool.colormap_normalize`.
+
+* :class:`pybedtools.plotting.BinaryHeatmap` class for working with results
+  from :meth:`BedTool.multi_intersect`.
+
+* :meth:`BedTool.each` now also has some filter capabilities (if provided
+  function's return value evaluates to False, feature will be skipped)
+
+* Better detection for samtools (thanks Luca Beltrame)
+
+* Expand BEDToolsError (thanks Ryan Layer)
+
+* Creating a BedTool from a list of intervals now saves to temp file instead of treating
+  like a consume-once iterator (#73)
+
+* Various fixes to keyword arg handling to match semantics of BEDTools.
+
+* Command line help and improved docs for the `peak_pie.py` script.
+
+* Fix to GFF attributes (thanks Libor Mořkovský)
+
+* Fix to labels in :mod:`pybedtools.contrib.venn_maker.py` (thanks Luca
+  Pinello)
+
+* Make the naive scaling (to million mapped reads) in
+  :func:`pybedtools.contrib.bigwig.bam_to_bigwiq` optional.
+
+* Fix for :meth:`BedTool.cat` to handle cases where at least one input is an
+  empty file
+
+* Removed SciPy dependency
+
+* Every commit is built with Travis-CI for continuous integration testing of
+  changes to source code.
+
+Changes in v0.6.2
+-----------------
+
+2012-11-05
+
+* Wrapped new tools available in BEDTools 2.17: :meth:`BedTool.jaccard` and
+  :meth:`BedTool.reldist` wrap the new `bedtools jaccard` and `bedtools
+  reldist` respectively.
+
+* Initial implementations of building blocks for computing statistics,
+  :meth:`BedTool.absolute_distance` and :meth:`BedTool.relative_distance`
+
+* :func:`pybedtools.featurefuncs.three_prime`,
+  :func:`pybedtools.featurefuncs.five_prime`, and
+  :func:`pybedtools.featurefuncs.TSS` modifier functions that can be passed to
+  :meth:`BedTool.each`
+
+* :func:`pybedtools.contrib.plotting.binary_heatmap` for visualizing results
+  from :meth:`BedTool.multi_intersect`
+
+* Fixed a long-standing issue where streaming :class:`BedTool` objects did not
+  close their open file handles (stdout).  When working with many (i.e. tens
+  of thousands) files, this caused the operating system to hit its open file
+  limit.  This is now fixed.
+
+* :meth:`BedTool.random_op`, a new mechanism for implementing operations that
+  you would like to apply over tens of thousands of shuffled interval files.
+  This makes it easy to extend the existing :mod:`pybedtools` multiprocessing
+  functionality.
+
+* :func:`pybedtools.contrib.bigwig.bam_to_bigwig`, a helper function to create
+  a libary-size-scaled bigWig file from an input BAM file.
+
+* :class:`pybedtools.contrib.plotting.TrackCollection` class, which handles
+  plotting multiple files at once, using a provided "stylesheet" configuration
+  to tweak colors etc.
+
+* :class:`pybedtools.contrib.plotting.BedToolsDemo` and
+  :class:`pybedtools.contrib.plotting.ConfiguredBedToolsDemo`, useful for
+  running many graphical demos of BEDTools operations using the same
+  "stylesheet" configuration.  Run :file:`pybedtools/contrib/plotting.py` for
+  a demo.
+
+* chromsizes dictionaries for common assemblies now have a `default` attribute,
+  which is an OrderedDict of a default set of chromosome.  For example,
+  ``pybedtools.chromsizes('hg19').default`` contains only the entries for the
+  autosomes and X and Y.
+
+* :meth:`BedTool.cat` now works better with multiprocessing
+
+* added `include_distribution` kwarg to :meth:`BedTool.randomstats`, which will
+  attach the full distribution of all the randomized files to the results
+  dictionary.
+
+* New method implementing Jaccard statistic (with pvalue using randomizations):
+  :meth:`BedTool.random_jaccard`
+
+* :func:`featurefuncs.extend_fields` helper function to pad fields with `'.'`,
+  useful for manipulating features with the :meth:`BedTool.each` method
+
+* Fixed a bug where BAM files, when written to disk via :meth:`BedTool.saveas`,
+  were saved as SAM files.
+
+* Better GTF/GFF detection, and if the input had quoted attribute values, then
+  the output will, too
+
+* various minor bug fixes and improvments as documented in the github commit
+  logs....
+
+
+Changes in v0.6.1
+-----------------
+
+2012-05-25
+
+* New :class:`pybedtools.contrib.plotting.Track` class allows plotting of
+  features with matplotlib.  The `Track` class subclasses
+  `matplotlib.collections.PolyCollection`, making it rather fast for 1000s of
+  features.
+
+* See the `scripts/pbt_plotting_example.py` script for a way of visually showing
+  the results of BEDTools operations . . . great for teaching BEDTools to new
+  users.
+
+* New :meth:`BedTool.liftover` method (needs a chain file from UCSC and the
+  `liftover` program installed)
+
+* :class:`BedTool` creation using tuples/lists of values -- everything is
+  converted to string before creating an :class:`Interval` object.
+
+* bugfix: :meth:`BedTool.window_maker` now handles the `genome` kwarg correctly
+
+* bugfix: `pybedtools.cleanup(remove_all=True)` now works correctly when using
+  the default temp dir
+
+
+Changes in v0.6
+---------------
+
+2012-03-13
+
+* Overhaul in online documentation to hopefully make functionality easier to
+  find and/or discover.  See :ref:`pybedtools reference` for summary tables of
+  the different parts of :mod:`pybedtools`; each entry is linked to further
+  class/method/function-specific docs.  These more detailed docs also have
+  links to view the source code from within the HTML docs for more exploration.
+
+* :func:`pybedtools.contrib.venn_maker` function that acts as an interface to
+  the VennDiagram R package -- just give it some BED files and it'll do the
+  rest.
+
+* Debug mode -- :func:`pybedtools.debug_mode` -- for verbose logging messages.
+
+* Fixed an open file leak (OSError: too many open files) that occured when
+  opening thousands of streaming bed files in a single session.
+
+* Initial support for tabix files.  Useful for extracting features from
+  a single region when you don't need a full intersection.
+
+* New :mod:`pybedtools.contrib` module (in the spirit of Django's `contrib`)
+  where higher-level functionality will be built.
+
+* :class:`pybedtools.contrib.Classifier` class for identifying the classes of
+  intervals.  Useful for making pie charts of intronic/exonic/intergenic etc
+  classes of peaks.  Note that this is somewhat redundant with the new `mapBed`
+  program in BEDTools.
+
+* Experimental :class:`pybedtools.contrib.IntersectionMatrix` class for
+  handling pairwise intersections of a large number of interval files --
+  including a local sqlite3 database to avoid re-computing already up-to-date
+  results.
+
+* :class:`Interval` objects are now hashable (it's just a hash of the string
+  representation) so that you can use them as dictionary keys.
+
+* :meth:`BedTool.split` method, which accepts a function returning an iterable
+  of :class:`Interval` objects. The function is applied to each interval.
+  Useful for, say, splitting each gene into TSS, TTS, upstream and downstream
+  features.
+
+* :meth:`BedTool.truncate_to_chrom` method, which truncates features to the
+  chromosome sizes of the provided genome.  Useful for when you try uploading
+  a MACS-generated track to the UCSC genome browser, but it complains because
+  peak boundaries have been extended outside chromosome boundaries . . . this
+  method fixes the problem.
+
+* :class:`BedTool` objects now have full functionality of :class:`IntervalFile`
+  objects -- that is, they have the methods :meth:`BedTool.any_hits`,
+  :meth:`BedTool.all_hits`, and :meth:`BedTool.count_hits` for doing
+  single-interval tests.  Sometimes this will be faster than using the tabix
+  support, sometimes it won't -- it's best to try both, depending on your data.
+
+* String representations of :class:`Interval` objects now have a newline at the
+  end, just like a raw lines from a BED/GFF/VCF file.  Previously, this was
+  inconsistent and sometimes led to extra blank lines in "streaming"
+  :class:`BedTool` instances . . . which in turn led to problems with BEDTools
+  programs using the chromsweep algorithm.
+
+* Concatentate multiple files with one call to :meth:`BedTool.cat` (thanks Jake
+  Biesinger)
+
+* Wrapped previous BEDTools programs:
+    * `unionBedGraphs` (:meth:`BedTool.union_bedgraphs`)
+    * `pairToBed` (:meth:`BedTool.pair_to_bed`)
+    * `pairToPair` (:meth:`BedTool.pair_to_pair`)
+    * `bedpeToBam` (:meth:`BedTool.bedpe_to_bam`)
+
+* Wrapped new BEDTools programs:
+    * `mapBed` (:meth:`BedTool.map`)
+    * `clusterBed` (:meth:`BedTool.cluster`)
+    * `randomBed` (:meth:`BedTool.random`)
+    * `multiIntersectBed` (:meth:`BedTool.multi_intersect`)
+    * `expandCols` (:meth:`BedTool.expand`)
+    * `windowMaker` (:meth:`BedTool.window_maker`)
+    * `bamToFastq` (:meth:`BedTool.bam_to_fastq`)
+
+* Made venn_gchart and venn_mpl tests more stable
+
+* Automatic documenting of which args are passed implicitly for BedTool method
+  calls
+
+* More robust mechanisms for specifying custom paths for BEDTools installation
+  as well as optional tabix, samtools, and R installations.  This makes it
+  easier to explicitly specify which versions of the tools to use.
+
+* Improvements to GFF attributes: handle unescaped "=" (from sim4db GFFs) and
+  make Attribute class properly dict-like (thanks Libor Mořkovský)
+
+Changes in v0.5.5
+-----------------
+
+2011-09-17
+
+* Use `additional_args` kwarg to pass arguments verbatim to the underlying
+  BEDTools programs.  This is necessary for arguments like
+  `genomeCoverageBed`'s `-5` argument, since `5=True` is not a valid Python
+  expression.  For example, you can use::
+
+     import pybedtools
+     a = pybedtools.example_bedtool('a.bed')
+     a.genome_coverage(bg=True, strand='+', genome='hg19', additional_args='-5')
+
+* Brent Pedersen added support for just 2 BED files in the Venn diagram scripts
+
+* :meth:`BedTool.all_hits` uses the underlying BEDTools C++ API to get all hits
+  in a file for a particular Interval::
+
+    a = pybedtools.example_bedtool('a.bed')
+    interval = Interval('chr1', 1, 5000)
+    a.all_hits(interval)
+
+* New semantics for comparisons of Interval objects.  Visual documentation of
+  this coming soon.
+
+* More tests for latest BEDTools code
+
+* Interval instances are now pickleable; they can now be used across processes
+  for parallel code.
+
+
+Changes in v0.5
+---------------
+
+2011-05-03
+
+* support for running random intersections in parallel.  See
+  :meth:`BedTool.randomstats` and :meth:`BedTool.randomintersection` (thanks,
+  Jake Biesinger)
+
+* Cython `Interval.__copy__()` for compatibility with `copy` module
+
+* `seek()` and `rewind()` methods for `IntervalFile` class, used for Aaron
+  Quinlan's new chromsweep algorithm (https://github.com/arq5x/chrom_sweep)
+  (thanks, Aaron)
+
+* support and tests for new BEDTools programs `multiBamCov`, `tagBam`, and `nucBed`
+
+* `output="out.bed"` kwarg for all wrapped methods for explicitly specifying
+  where to save output -- no more moving tempfiles
+
+* docs improvements:
+    * direct comparison with a shell script to illustrate benefit of
+      `pybedtools`; see :ref:`shell_comparison`
+    * more installation details
+    * 0- and 1-based coordinates discussed early on (the 3 brief examples page,
+      :ref:`3examples`)
+    * development history and open collaboration model (see :ref:`devmodel`)

pybedtools/source/docs/source/conf.py

@@ -0,0 +1,237 @@
+# -*- coding: utf-8 -*-
+#
+# pybedtools documentation build configuration file, created by
+# sphinx-quickstart on Wed Dec 22 17:39:12 2010.
+#
+# This file is execfile()d with the current directory set to its containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys, os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.insert(0, os.path.abspath('../..'))
+
+from pybedtools import __version__ as version
+
+# -- General configuration -----------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be extensions
+# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
+extensions = ['sphinx.ext.autodoc', 'sphinx.ext.autosummary',
+              'sphinx.ext.doctest', 'sphinx.ext.todo',
+              'sphinx.ext.coverage','sphinx.ext.viewcode', 'numpydoc']
+
+doctest_test_doctest_blocks = 'default'
+
+# From http://stackoverflow.com/questions/12206334/\
+# sphinx-autosummary-toctree-contains-refere\
+# nce-to-nonexisting-document-warnings
+numpydoc_show_class_members = False
+
+# this is needed to get the autodoc_source.rst doctests to run
+doctest_global_setup = """
+from pybedtools import *
+import pybedtools
+"""
+
+autosummary_generate = True
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix of source filenames.
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = 'pybedtools'
+copyright = '2010-2015, Ryan Dale'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = version
+# The full version, including alpha/beta/rc tags.
+release = version
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = []
+
+# The reST default role (used for this markup: `text`) to use for all documents.
+default_role = 'file'
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+#pygments_style = 'sphinx'
+highlight_language = 'python'
+html_use_smartypants = False
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+
+# -- Options for HTML output ---------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#html_theme = 'nature'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = ['_themes']
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = ''
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'pybedtoolsdoc'
+
+
+# -- Options for LaTeX output --------------------------------------------------
+
+# The paper size ('letter' or 'a4').
+#latex_paper_size = 'letter'
+
+# The font size ('10pt', '11pt' or '12pt').
+#latex_font_size = '10pt'
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title, author, documentclass [howto/manual]).
+latex_documents = [
+  ('index', 'pybedtools.tex', 'pybedtools Documentation',
+   'Ryan Dale', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+latex_use_parts = True
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Additional stuff for the LaTeX preamble.
+#latex_preamble = ''
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output --------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    ('index', 'pybedtools', 'pybedtools Documentation',
+     ['Ryan Dale'], 1)
+]

pybedtools/source/docs/source/create-a-bedtool-tutorial.rst

@@ -0,0 +1,39 @@
+.. include:: includeme.rst
+
+Create a :class:`BedTool`
+-------------------------
+First, follow the :ref:`installation` instructions if you haven't already
+done so to install both BEDTools_ and :mod:`pybedtools`.
+
+Then import the :mod:`pybedtools` module and make a new :class:`BedTool`.  A
+:class:`BedTool` object encapsulates all of the available BEDTools programs and
+makes them easier to use within Python.  Most of the time when working with
+:mod:`pybedtools` you'll be using :class:`BedTool` objects.  In general, a
+single :class:`BedTool` object points to an interval file (BED, GFF, GTF, VCF,
+SAM, or BAM format).
+
+::
+
+    >>> import pybedtools
+
+    >>> # use a BED file that ships with pybedtools...
+    >>> a = pybedtools.example_bedtool('a.bed')
+
+    >>> # ...or use your own by passing a filename
+    >>> a = pybedtools.BedTool('peaks.bed')
+
+This documentation uses example files that ship with :mod:`pybedtools`.  To
+access these files from their installation location, we use the
+:func:`example_bedtool` function.  This is convenient because if you copy-paste
+the examples, they will work. When using the :func:`example_bedtool` function,
+the resulting :class:`BedTool` object will point to the corresponding file in
+the `test/data` directory of your :mod:`pybedtools` installation.  If you would
+rather learn using your own files, just pass the filename to a new
+:class:`BedTool`, like the above example.
+
+You can use any file that BEDTools_ supports -- this includes BED, VCF,
+GFF, and gzipped versions of any of these. See :ref:`Creating a BedTool`
+for more on the different ways of creating a :class:`BedTool`, including
+from iterators and directly from a string.
+
+Now, let's see how to do a common task performed on BED files: intersections.

pybedtools/source/docs/source/default-arguments.rst

@@ -0,0 +1,81 @@
+.. currentmodule:: pybedtools
+
+
+Default arguments
+=================
+Recall in the earlier :ref:`intersections` section that we passed the `u=True` argument to :meth:`a.intersect`:
+
+.. doctest::
+
+    >>> import pybedtools
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> b = pybedtools.example_bedtool('b.bed')
+    >>> a_with_b = a.intersect(b, u=True)
+
+Let's do the same thing but use different variable names for the :class:`BedTool` objects so that
+the next section is less confusing:
+
+.. doctest::
+
+    >>> import pybedtools
+    >>> exons = pybedtools.example_bedtool('a.bed')
+    >>> snps = pybedtools.example_bedtool('b.bed')
+    >>> exons_with_snps = exons.intersect(snps, u=True)
+
+
+While we're on the subject of arguments, note that we didn't have to specify
+`-a` or `-b` arguments, like you would need if calling `intersectBed` from the
+command line.  In other words, since `exons` refers to the file `a.bed` and
+`snps` refers to the file `b.bed`, the following line::
+
+    >>> exons_with_snps = exons.intersect(snps, u=True)
+
+is equivalent to the command line usage of::
+
+    $ intersectBed -a a.bed -b b.bed -u > tmpfile
+
+But we didn't have to explicitly pass the argument for `-a` because
+:class:`BedTool` objects make some assumptions for convenience.
+
+We're calling a method on the :class:`BedTool` object `exons`, so
+:mod:`pybedtools` assumes that the file `exons` points to (stored in the
+attribute `exons.fn`) is the one we want to use as input.  So by default, we
+don't need to explicitly give the keyword argument `a=exons.fn` because the
+:meth:`exons.intersect` method does so automatically.
+
+We're also calling a method that takes a second bed file as input  -- other
+such methods include :meth:`BedTool.subtract` and :meth:`BedTool.closest`,
+and others.  For these methods, in addition to assuming `-a` is taken care
+of by the :attr:`BedTool.fn` attribute, :mod:`pybedtools` also assumes the
+first unnamed argument to these methods are the second file you want to
+operate on (and if you pass a :class:`BedTool`, it'll automatically use the
+file in the `fn` attribute of that :class:`BedTool`).
+
+An example may help to illustrate: these different ways of calling
+:meth:`BedTool.intersect` all have the same results, with the first version
+being the most compact (and probably most convenient):
+
+.. doctest::
+
+    >>> # these all have identical results
+    >>> x1 = exons.intersect(snps)
+    >>> x2 = exons.intersect(a=exons.fn, b=snps.fn)
+    >>> x3 = exons.intersect(b=snps.fn)
+    >>> x4 = exons.intersect(snps, a=exons.fn)
+    >>> x1 == x2 == x3 == x4
+    True
+
+Note that `a.intersect(a=a.fn, b)` is not a valid Python expression, since
+non-keyword arguments must come before keyword arguments, but
+`a.intersect(b, a=a.fn)` works fine.
+
+If you're ever unsure, the docstring for these methods indicates which, if
+any, arguments are used as default.  For example, in the
+:meth:`BedTool.intersect` help, it says::
+
+    For convenience, the file or stream this BedTool points to is implicitly
+    passed as the -a argument to intersectBed
+
+OK, enough about arguments for now, but you can read more about them in
+:ref:`similarity principle`, :ref:`default args principle` and :ref:`non
+defaults principle`.

pybedtools/source/docs/source/each.rst

@@ -0,0 +1,66 @@
+.. include:: includeme.rst
+
+Each
+====
+Similar to :meth:`BedTool.filter`, which applies a function to return True
+or False given an :class:`Interval`, the :meth:`BedTool.each` method applies a
+function to return a new, possibly modified :class:`Interval`.
+
+The :meth:`BedTool.each` method applies a function to every feature.  Like
+:meth:`BedTool.filter`, you can use your own function or some pre-defined
+ones in the :mod:`featurefuncs` module.  Also like :meth:`filter`, `*args`
+and `**kwargs` are sent to the function.
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> b = pybedtools.example_bedtool('b.bed')
+
+    >>> # The results of an "intersect" with c=True will return features
+    >>> # with an additional field representing the counts.
+    >>> with_counts = a.intersect(b, c=True)
+
+Let's define a function that will take the number of counts in each feature
+as calculated above and divide by the number of bases in that feature.  We
+can also supply an optional scalar, like 0.001, to get the results in
+"number of intersections per kb".  We then insert that value into the score
+field of the feature.  Here's the function:
+
+.. doctest::
+
+    >>> def normalize_count(feature, scalar=0.001):
+    ...     """
+    ...     assume feature's last field is the count
+    ...     """
+    ...     counts = float(feature[-1])
+    ...     normalized = round(counts / (len(feature) * scalar), 2)
+    ...
+    ...     # need to convert back to string to insert into feature
+    ...     feature.score = str(normalized)
+    ...     return feature
+
+And we apply it like this:
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> normalized = with_counts.each(normalize_count)
+    >>> print(normalized)
+    chr1	1	100	feature1	0.0	+	0
+    chr1	100	200	feature2	10.0	+	1
+    chr1	150	500	feature3	2.86	-	1
+    chr1	900	950	feature4	20.0	+	1
+    <BLANKLINE>
+
+Similar to :meth:`BedTool.filter`, we could have used the Python built-in
+function `map` to map a function to each :class:`Interval`.  In fact, this can
+still be useful if you don't want a :class:`BedTool` object as a result.  For
+example::
+
+    >>> feature_lengths = map(len, a)
+
+However, the :meth:`BedTool.each` method returns a :class:`BedTool` object,
+which can be used in a chain of commands, e.g., ::
+
+    >>> a.intersect(b).each(normalize_count).filter(lamda x: float(x[4]) < 1e-5)

pybedtools/source/docs/source/example-script

@@ -0,0 +1,107 @@
+
+import pybedtools
+
+# Create a BedTool for the GFF file of annotations
+g = pybedtools.BedTool('example.gff')
+
+
+# Set up two functions that will filter and then rename features to set up for
+# merging
+
+
+def renamer(x):
+    """
+    *x* is an Interval object representing a GFF feature.
+
+    Renames the feature after the feature type; this is needed for when
+    .merge() combines names together in a later step. 
+    """
+
+    # This illustrates setting and getting fields in an Interval object based
+    # on attribute or index
+    x.name = x[2]
+    return x
+
+
+def filter_func(x):
+    """
+    *x* is an Interval object representing a GFF feature.
+
+    This filter function will only pass features of type "intron" or "exon"
+    """
+    if x[2] in ('intron', 'exon'):
+        return True
+    return False
+
+
+# Filter and rename the GFF features by passing the above functions to
+# .filter() and .each(). Note that since each method returns a new BedTool,
+# methods can be chained together
+g2 = g.filter(filter_func).each(renamer)
+
+# Save a copy of the new GFF file for later inspection
+g2 = g2.saveas('edited.gff')
+
+
+# Here we call mergeBed, which operates on the file pointed to by g2
+# (that is, 'edited.gff').
+#
+# We use several options for BEDTools mergeBed: 
+#
+#   `nms` combines names of merged features (after filtering and renaming, this
+#   is either "intron" or "exon") into a semicolon-delimited list;
+#
+#   d=-1 does not merge bookended features together; 
+#
+#   s=True ensures a stranded merge;
+#
+#   scores='sum' ensures a valid BED file result, with a score field before the
+#   strand field
+#
+merged = g2.merge(nms=True, d=-1, s=False, scores='sum')
+
+# Next, we intersect a BAM file with the merged features.  Here, we explicitly
+# specify the `abam` and `b` arguments, ensure stranded intersections, use
+# BED-format output, and report the entire a and b features in the output:
+#
+reads_in_features = merged.intersect(abam='example.bam',
+                                     b=merged.fn,
+                                     s=True, 
+                                     bed=True,
+                                     wao=True)
+
+# Set up a dictionary to hold counts
+from collections import defaultdict
+results = defaultdict(int)
+
+# Iterate through the intersected reads, parse out the names of the features
+# they intersected, and increment counts in the dictionary. This illustrates
+# how BedTool objects follow the iterator protocol, each time yielding an
+# Interval object:
+#
+total = 0.0
+for intersected_read in reads_in_features:
+    total += 1
+
+    # Extract the name of the feature this read intersected by indexing into
+    # the Interval
+    intersected_feature = feature[-4]
+
+    # Convert names like "intron;intron;intron", which indicates overlapping
+    # isoforms or genes all with introns in this region, to the simple class of
+    # "intron"
+    key = ';'.join(sorted(list(set(intersected_with.split(';')))))
+
+    # Increment the count for this class
+    results[key] += 1
+
+# Rename the "." key to something more meaningful
+results['intergenic'] = results.pop('.')
+
+# Add the total to the dictionary
+results['total'] = int(total)
+
+print results
+
+# Delete any temporary files created
+pybedtools.cleanup()

pybedtools/source/docs/source/example-script-nocomments

@@ -0,0 +1,37 @@
+import pybedtools
+
+g = pybedtools.BedTool('example.gff')
+
+def renamer(x):
+    x.name = x[2]
+    return x
+
+def filter_func(x):
+    if x[2] in ('intron', 'exon'):
+        return True
+    return False
+
+
+g2 = g.filter(filter_func).each(renamer)
+g2 = g2.saveas('edited.gff')
+merged = g2.merge(nms=True, d=-1, s=False, scores='sum')
+reads_in_features = merged.intersect(abam='example.bam',
+                                     b=merged.fn,
+                                     s=True, 
+                                     bed=True,
+                                     wao=True)
+
+from collections import defaultdict
+results = defaultdict(int)
+total = 0.0
+for intersected_read in reads_in_features:
+    total += 1
+    intersected_feature = feature[-4]
+    key = ';'.join(sorted(list(set(intersected_with.split(';')))))
+    results[key] += 1
+
+results['intergenic'] = results.pop('.')
+results['total'] = int(total)
+print results
+
+pybedtools.cleanup()

pybedtools/source/docs/source/example_3

@@ -0,0 +1,109 @@
+#!/usr/bin/env python
+
+"""
+Example from pybedtools documentation: find reads in introns and exons using
+multiple CPUs.
+
+Prints a tab-separated file containing class (exon, intron, both) and number of
+reads in each class.
+"""
+import pybedtools
+import argparse
+import os
+import sys
+import multiprocessing
+
+if __name__ == "__main__":
+
+    ap = argparse.ArgumentParser(prog=os.path.basename(sys.argv[0]), usage=__doc__)
+    ap.add_argument(
+        "--gff", required=True, help="GFF or GTF file containing annotations"
+    )
+    ap.add_argument(
+        "--bam", required=True, help="BAM file containing reads to be counted"
+    )
+    ap.add_argument(
+        "--stranded",
+        action="store_true",
+        help="Use strand-specific merging and overlap. " "Default is to ignore strand",
+    )
+    ap.add_argument(
+        "--processes",
+        default=1,
+        type=int,
+        help="Number of processes to use in parallel.",
+    )
+    ap.add_argument(
+        "-v", "--verbose", action="store_true", help="Verbose (goes to stderr)"
+    )
+    args = ap.parse_args()
+
+    gff = args.gff
+    bam = args.bam
+    stranded = args.stranded
+
+    if args.processes > 3:
+        print(
+            "Only need 3 processes (one each for exon, intron, both), so "
+            "resetting processes from {0} to 3".format(args.processes)
+        )
+        args.processes = 3
+
+    def featuretype_filter(feature, featuretype):
+        """
+        Only passes features with the specified *featuretype*
+        """
+        if feature[2] == featuretype:
+            return True
+        return False
+
+    def subset_featuretypes(featuretype):
+        """
+        Returns the filename containing only `featuretype` features.
+        """
+        return g.filter(featuretype_filter, featuretype).saveas().fn
+
+    def count_reads_in_features(features):
+        """
+        Callback function to count reads in features
+        """
+        return (
+            pybedtools.BedTool(bam).intersect(
+                features, s=stranded, bed=True, stream=True
+            )
+        ).count()
+
+    # Some GFF files have invalid entries -- like chromosomes with negative coords
+    # or features of length = 0.  This line removes them (the `remove_invalid`
+    # method) and saves the result in a tempfile
+    g = pybedtools.BedTool(gff).remove_invalid().saveas()
+
+    # Set up pool of workers
+    with multiprocessing.Pool(processes=args.processes) as pool:
+
+        # Get separate files for introns and exons in parallel
+        featuretypes = ["intron", "exon"]
+        introns, exons = pool.map(subset_featuretypes, featuretypes)
+
+        # Since `subset_featuretypes` returns filenames, we convert to BedTool objects
+        # to do intersections below.
+        introns = pybedtools.BedTool(introns)
+        exons = pybedtools.BedTool(exons)
+
+        # Identify unique and shared regions using bedtools commands subtract, merge,
+        # and intersect.
+        exon_only = exons.subtract(introns).merge()
+        intron_only = introns.subtract(exons).merge()
+        intron_and_exon = exons.intersect(introns).merge()
+
+        # Do intersections with BAM file in parallel. Note that we're passing filenames
+        # to multiprocessing.Pool rather than BedTool objects.
+        features = (exon_only.fn, intron_only.fn, intron_and_exon.fn)
+
+        # Run count_reads_in_features in parallel over features
+        results = pool.map(count_reads_in_features, features)
+
+        labels = ("exon_only", "intron_only", "intron_and_exon")
+
+    for label, reads in zip(labels, results):
+        print("{0}\t{1}".format(label, reads))

pybedtools/source/docs/source/example_3_no_comments

@@ -0,0 +1,49 @@
+import sys
+import multiprocessing
+import pybedtools
+
+gff = pybedtools.example_filename('gdc.gff')
+bam = pybedtools.example_filename('gdc.bam')
+
+g = pybedtools.BedTool(gff).remove_invalid().saveas()
+
+
+def featuretype_filter(feature, featuretype):
+    if feature[2] == featuretype:
+        return True
+    return False
+
+
+def subset_featuretypes(featuretype):
+    result = g.filter(featuretype_filter, featuretype).saveas()
+    return pybedtools.BedTool(result.fn)
+
+
+def count_reads_in_features(features_fn):
+    """
+    Callback function to count reads in features
+    """
+
+    return pybedtools.BedTool(bam).intersect(
+                             b=features_fn,
+                             stream=True).count()
+
+
+pool = multiprocessing.Pool()
+
+featuretypes = ('intron', 'exon')
+introns, exons = pool.map(subset_featuretypes, featuretypes)
+
+exon_only = exons.subtract(introns).merge().remove_invalid().saveas().fn
+intron_only = introns.subtract(exons).merge().remove_invalid().saveas().fn
+intron_and_exon = exons.intersect(introns).merge().remove_invalid().saveas().fn
+
+features = (exon_only, intron_only, intron_and_exon)
+results = pool.map(count_reads_in_features, features)
+
+labels = ('      exon only:',
+          '    intron only:',
+          'intron and exon:')
+
+for label, reads in zip(labels, results):
+    sys.stdout.write('%s %s\n' % (label, reads))

pybedtools/source/docs/source/filtering.rst

@@ -0,0 +1,93 @@
+
+.. include:: includeme.rst
+
+.. _filtering:
+
+Filtering
+~~~~~~~~~
+The :meth:`BedTool.filter` method lets you pass in a function that accepts an
+:class:`Interval` as its first argument and returns True for False.  This
+allows you to perform "grep"-like operations on :class:`BedTool` objects.  For
+example, here's how to get a new :class:`BedTool` containing features from `a`
+that are more than 100 bp long:
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> b = a.filter(lambda x: len(x) > 100)
+    >>> print(b)
+    chr1	150	500	feature3	0	-
+    <BLANKLINE>
+
+The :meth:`filter` method will pass its `*args` and `**kwargs` to the function
+provided.  So here is a more generic case, where the function is defined once
+and different arguments are passed in for filtering on different lengths:
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> def len_filter(feature, L):
+    ...     "Returns True if feature is longer than L"
+    ...     return len(feature) > L
+
+Now we can pass different lengths without defining a new function for each
+length of interest, like this:
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+
+    >>> print(a.filter(len_filter, L=10))
+    chr1	1	100	feature1	0	+
+    chr1	100	200	feature2	0	+
+    chr1	150	500	feature3	0	-
+    chr1	900	950	feature4	0	+
+    <BLANKLINE>
+
+    >>> print(a.filter(len_filter, L=99))
+    chr1	100	200	feature2	0	+
+    chr1	150	500	feature3	0	-
+    <BLANKLINE>
+
+    >>> print(a.filter(len_filter, L=200))
+    chr1	150	500	feature3	0	-
+    <BLANKLINE>
+
+
+See :ref:`BedTools as iterators` for more advanced and space-efficient usage
+of :meth:`filter` using iterators.
+
+Note that we could have used the built-in Python function, `filter()`, but that
+would have returned an iterator that we would have to construct a new
+:class:`pybedtools.BedTool` out of.  The :meth:`BedTool.filter` method returns
+a ready-to-use :class:`BedTool` object, which allows embedding of
+:meth:`BedTool.filter` calls in a chain of commands, e.g.::
+
+    >>> a.intersect(b).filter(lambda x: len(x) < 100).merge()
+
+Fast filtering functions in Cython
+----------------------------------
+
+The :mod:`featurefuncs` module contains some ready-made functions written
+in Cython that will be faster than pure Python equivalents.  For example,
+there are :func:`greater_than` and :func:`less_than` functions, which are
+about 70% faster.  In IPython::
+
+    >>> from pybedtools.featurefuncs import greater_than
+
+    >>> len(a)
+    310456
+
+    >>> def L(x,width=100):
+    ...     return len(x) > 100
+
+    >>> # The %timeit command is from IPython, and won't work
+    >>> # in a regular Python script:
+    >>> %timeit a.filter(greater_than, 100)
+    1 loops, best of 3: 1.74 s per loop
+
+    >>> %timeit a.filter(L, 100)
+    1 loops, best of 3: 2.96 s per loop
+

pybedtools/source/docs/source/flow-of-commands.rst

@@ -0,0 +1,120 @@
+Under the hood
+==============
+
+This section documents some details about what happens when a :class:`BedTool`
+object is created and exactly what happens when a BEDTools command is called.
+It's mostly useful for developers or for debugging.
+
+
+There are three kinds of sources/sinks for BedTool objects:
+
+* filename
+* open file object
+* iterator of Interval objects
+
+
+Iterator "protocol"
+-------------------
+BedTool objects yield an Interval object on each `next()` call.  Where this
+Interval comes from depends on how the BedTool was created and what format the
+underlying data are in, as follows.
+
+Filename-based
+~~~~~~~~~~~~~~
+If BED/GTF/GFF/VCF format, then use an `IntervalFile` object for Cython/C++
+speed.
+
+If SAM format, then use an `IntervalIterator`.  This is a Cython object that
+reads individual lines and passes them to `create_interval_from_list`, a Cython
+function.  `create_interval_from_list` does a lot of the work to figure out
+what format the line is, and this is how we are able to support SAM Interval
+objects.
+
+If BAM format, then first do a Popen call to `samtools view`, and create an
+`IntervalIterator` from subprocess.PIPE similar to SAM format.
+
+Open file-based
+~~~~~~~~~~~~~~~
+All formats are passed to an `IntervalIterator`, which reads one line at
+a time and yields an `Interval` object.
+
+If it's a BAM file (specifically, a detected bgzip stream), then it's actually
+first sent to the stdin of a `samtools` Popen call, and then the
+subprocess.PIPE from that Popen's stdout is sent to an `IntervalIterator`.
+
+Iterator or generator-based
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+If it's neither of the above, then the assumption is that it's already an
+iterable of `Interval` objects.  This is the case if a `BedTool` is created
+with something like::
+
+    a = pybedtools.example_bedtool('a.bed')
+    b = pybedtools.BedTool((i for i in a))
+
+
+In this case, the `(i for i in a)` creates a generator of intervals from an
+`IntervalFile` -- since `a` is a filename-based BedTool.  Since the first
+argument to the BedTool constructor is neither a filename nor an open file, the
+new BedTool `b`'s `.fn` attribute is directly set to this generator . . . so we
+have a generator-based BedTool.
+
+Calling BEDTools programs
+-------------------------
+Depending on the type of BedTool (filename, open file, or iterator), the method
+of calling BEDTools programs differs.
+
+In all cases, BEDTools commands are called via a `subprocess.Popen` call
+(hereafter called "the Popen" for convenience).  Depending on the type of
+BedTool objects being operated on, the Popen will be passed different objects
+as stdin and/or stdout.
+
+In general, using a filename as input is the most straightforward -- nothing is
+passed to the Popen's stdin because the filenames are embedded in the BEDTools
+command.
+
+Using non-filename-based BedTools means that they are passed, one line at
+a time, to the stdin of the Popen.  The commands for the BEDTools call
+will specify "stdin" in these cases, as is standard for the BEDTools suite.
+
+The default is for the output to be file-based.  In this case, an open tempfile
+object is provided as the Popen's stdout.
+
+If the returned BedTool is requested to be a "streaming" BedTool, then the
+Popen's stdout will be subprocess.PIPE, and the new BedTool object will be
+open-file based (which is what subprocess.PIPE acts like).
+
+Specifically, here is the information flow of stdin/stdout for various
+interconversions of BedTool types . . . .
+
+
+:filename -> filename:
+    The calling BedTool is filename-based and `stream=False`.
+
+    * `stdin`: `None` (the filenames are provided in the BEDTools command)
+    * `stdout`: open tempfile object
+    * new BedTool: filename-based BedTool pointing to the tempfile's filename
+
+:filename -> open file object:
+    The calling BedTool is filename-based and `stream=True` is requested.
+
+    * `stdin`: None (provided in the cmds)
+    * `stdout`: open file object -- specifically, subprocess.PIPE
+    * new BedTool: iterator-based BedTool.  Each `next()` call retrieves the
+      next line in subprocess.PIPE
+
+:open file object -> filename:
+    The calling BedTool is from, e.g., subprocess.PIPE and there's
+    a saveas() call to "render" to file.
+
+    * `stdin`: each line in the open file object is written to subprocess.PIPE
+    * `stdout`: open file object -- either a tempfile or new file created from
+      supplied filename
+    * new BedTool: filename-based BedTool
+
+:open file object -> iterator:
+    The calling BedTool is usually based on subprocess.PIPE, and the output
+    will *also* come from subprocess.PIPE.
+
+    * `stdin`: each line from the open file is written to subprocess.PIPE
+    * `stdout`: open file object, subprocess.PIPE
+    * new BedTool: filename based on subprocess.PIPE

pybedtools/source/docs/source/history.rst

@@ -0,0 +1,141 @@
+.. include:: includeme.rst
+
+.. _`working with history`:
+
+Using the history and tags
+--------------------------
+`BEDTools`_ makes it very easy to do rather complex genomic algebra.  Sometimes
+when you're doing some exploratory work, you'd like to rewind back to a
+previous step, or clean up temporary files that have been left on disk over the
+course of some experimentation.
+
+To assist this sort of workflow, :class:`BedTool` instances keep track of
+their history in the :attr:`BedTool.history` attribute.  Let's make an
+example :class:`BedTool`, `c`, that has some history:
+
+.. doctest::
+    :options: +NORMALIZE_WHITESPACE
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> b = pybedtools.example_bedtool('b.bed')
+    >>> c = a.intersect(b, u=True)
+
+
+`c` now has a history which tells you all sorts of useful things (described
+in more detail below)::
+
+    >>> print c.history
+    [<HistoryStep> bedtool("/home/ryan/pybedtools/pybedtools/test/a.bed").intersect("/home/ryan/pybedtools/pybedtools/test/b.bed", u=True), parent tag: klkreuay, result tag: egzgnrvj]
+
+
+There are several things to note here.  First, the history describes the full
+commands, including all the names of the temp files and all the arguments that
+you would need to run in order to re-create it.  Since :class:`BedTool` objects
+are fundamentally file-based, the command refers to the underlying filenames
+(i.e., :file:`a.bed` and :file:`b.bed`) instead of the :class:`BedTool`
+instances (i.e., `a` and `b`). A simple copy-paste of the command will be
+enough re-run the command. While this may be useful in some situations, be
+aware that if you do run the command again you'll get *another* temp file that
+has the same contents as `c`'s temp file.
+
+To avoid such cluttering of your temp dir, the history also reports
+**tags**. :class:`BedTool` objects, when created, get a random tag assigned
+to them.  You can get get the :class:`BedTool` associated with tag with the
+:func:`pybedtools.find_tagged` function. These tags are used to keep track
+of instances during this session.
+
+So in this case, we could get a reference to the `a` instance with::
+
+    >>> should_be_a = pybedtools.find_tagged('klkreuay')
+
+Here's confirmation that the parent of the first step of `c`'s history is
+`a` (note that :class:`HistoryStep` objects have a
+:attr:`HistoryStep.parent_tag` and :attr:`HistoryStep.result_tag`):
+
+.. doctest::
+
+    >>> pybedtools.find_tagged(c.history[0].parent_tag) == a
+    True
+
+Let's make something with a more complicated history:
+
+.. doctest::
+
+    >>> a = pybedtools.example_bedtool('a.bed')
+    >>> b = pybedtools.example_bedtool('b.bed')
+    >>> c = a.intersect(b)
+    >>> d = c.slop(g=pybedtools.chromsizes('hg19'), b=1)
+    >>> e = d.merge()
+
+    >>> # this step adds complexity!
+    >>> f = e.subtract(b)
+
+Let's see what the history of `f` (the last :class:`BedTool` created) looks
+like . . . note that here I'm formatting the results to make it easier to
+see::
+
+    >>> print f.history
+    [
+    |   [
+    |   |   [
+    |   |   |   [
+    |   |   |   |<HistoryStep> BedTool("/usr/local/lib/python2.6/dist-packages/pybedtools/test/data/a.bed").intersect(
+    |   |   |   |                      "/usr/local/lib/python2.6/dist-packages/pybedtools/test/data/b.bed", 
+    |   |   |   |                      ), 
+    |   |   |   |                      parent tag: rzrztxlw, 
+    |   |   |   |                      result tag: ifbsanqk
+    |   |   |   ],
+    |   |   |
+    |   |   |<HistoryStep> BedTool("/tmp/pybedtools.BgULVj.tmp").slop(
+    |   |   |                      b=1,genome="hg19"
+    |   |   |                      ), 
+    |   |   |                      parent tag: ifbsanqk, 
+    |   |   |                      result tag: omfrkwjp
+    |   |   ],
+    |   |
+    |   |<HistoryStep> BedTool("/tmp/pybedtools.SFmbYc.tmp").merge(),
+    |   |                      parent tag: omfrkwjp,
+    |   |                      result tag: zlwqblvk
+    |   ], 
+    |
+    |<HistoryStep> BedTool("/tmp/pybedtools.wlBiMo.tmp").subtract(
+    |                      "/usr/local/lib/python2.6/dist-packages/pybedtools/test/data/b.bed",
+    |                      ),
+    |                      parent tag: zlwqblvk, 
+    |                      result tag: reztxhen
+    ]
+
+Those first three history steps correspond to `c`, `d`, and `e`
+respectively, as we can see by comparing the code snippet above with the
+commands in each history step.  In other words, `e` can be described by the
+sequence of 3 commands in the first three history steps.  In fact, if we
+checked `e.history`, we'd see exactly those same 3 steps.
+
+When `f` was created above, it operated both on `e`, which had its own
+history, as well as `b` -- note the nesting of the list. You can do
+arbitrarily complex "genome algebra" operations, and the history of the
+:class:`BEDTools` will keep track of this.  It may not be useful in every
+situtation, but the ability to backtrack and have a record of what you've
+done can sometimes be helpful.
+
+Deleting temp files specific to a single :class:`BedTool`
+---------------------------------------------------------
+You can delete temp files that have been created over the history of a
+:class:`BedTool` with :meth:`BedTool.delete_temporary_history`.  This method
+will inspect the history, figure out which items point to files in the temp dir
+(which you can see with :func:`get_tempdir`), and prompt you for their
+deletion::
+
+    >>> f.delete_temporary_history()
+    Delete these files?
+        /tmp/pybedtools..BgULVj.tmp
+        /tmp/pybedtools.SFmbYc.tmp
+        /tmp/pybedtools.wlBiMo.tmp
+    (y/N) y
+
+Note that the file that `f` points to is left alone.  To clarify, the
+:meth:`BedTool.delete_temporary_history` will only delete temp files that match
+the pattern ``<TEMP_DIR>/pybedtools.*.tmp`` from the history of `f`, up to but
+not including the file for `f` itself.  Any :class:`BedTool` instances that do
+not match the pattern are left alone.  Use the kwarg `ask=False` to disable
+the prompt.

pybedtools/source/docs/source/includeme.rst

@@ -0,0 +1,50 @@
+
+.. currentmodule:: pybedtools
+
+.. _Tabix: http://samtools.sourceforge.net/tabix.shtml
+
+.. _download page: http://sourceforge.net/projects/samtools/files/
+
+.. _samtools: http://samtools.sourceforge.net/
+
+.. _tempdir: http://docs.python.org/library/tempfile.html#tempfile.tempdir
+
+.. _filo: https://github.com/arq5x/filo
+
+.. _R: http://www.r-project.org/
+
+.. _BEDTools: http://github.com/arq5x/bedtools
+
+.. _BEDTools documentation: http://code.google.com/p/bedtools/#Documentation
+
+.. _Learn Python the Hard Way: http://learnpythonthehardway.org/static/LearnPythonTheHardWay.pdf
+
+.. _IPython: http://ipython.scipy.org/moin/
+
+.. _BED format: http://genome.ucsc.edu/FAQ/FAQformat#format1
+
+.. _pip: http://www.pip-installer.org/en/latest/installing.html
+
+.. _easy_install: http://pypi.python.org/pypi/setuptools
+
+.. _Python Package Index: http://pypi.python.org/pypi
+
+.. _Cython: http://cython.org/
+
+.. _Python: http://www.python.org/
+
+.. _nosetests: http://somethingaboutorange.com/mrl/projects/nose/
+
+.. _PyYAML: http://pyyaml.org/wiki/PyYAMLDocumentation
+
+.. _Sphinx: http://sphinx.pocoo.org/
+
+.. _Cygwin: http://www.cygwin.com
+
+.. _argparse: http://pypi.python.org/pypi/argparse
+
+.. _nose: http://pypi.python.org/pypi/nose
+
+.. _scipy: http://www.scipy.org/
+
+.. _matplotlib: http://matplotlib.sourceforge.net/

pybedtools/source/docs/source/index.rst

@@ -0,0 +1,68 @@
+.. pybedtools documentation master file, created by
+   sphinx-quickstart on Wed Dec 22 17:39:12 2010.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+.. include:: includeme.rst
+
+`pybedtools` documentation
+==========================
+
+.. include:: ../../README.rst
+
+
+
+As of 2022, `pybedtools` is released under the MIT license; see LICENSE.txt for
+more info.
+
+
+.. note::
+
+    If you use :mod:`pybedtools` in your work, please cite the `pybedtools
+    manuscript <http://bioinformatics.oxfordjournals.org/content/27/24/3423>`_
+    and the `BEDTools manuscript
+    <http://bioinformatics.oxfordjournals.org/content/26/6/841.short>`_:
+
+        Dale RK, Pedersen BS, and Quinlan AR. 2011. *Pybedtools: a flexible
+        Python library for manipulating genomic datasets and annotations*.
+        Bioinformatics 27(24):3423-3424.
+
+        Quinlan AR and Hall IM, 2010. *BEDTools: a flexible suite of utilities
+        for comparing genomic features*. Bioinformatics 26(6):841–842.
+
+
+Getting started
+---------------
+
+The documentation is separated into 4 main parts, depending on the depth you'd
+like to cover:
+
+* Lazy, or just want to jump in?  Check out :ref:`3examples` to
+  get a feel for the package.
+* Want a guided tour?  Give the :ref:`tutorial` a shot.
+* More advanced features are described in the :ref:`topical` section. 
+* Finally, doctested module documentation can be found in :ref:`autodoc`.
+
+
+Contents:
+---------
+
+.. toctree::
+   :maxdepth: 2
+
+   main
+   3-brief-examples
+   tutorial-contents
+   topical-documentation-contents
+   FAQs
+   scripts
+   autodoc_source
+   changes
+
+
+Indices and tables
+==================
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+