Package: pdfplumber-aemc
pdfplumberPlumb a PDF for detailed information about each text character, rectangle, and line. Plus: Table extraction and visual debugging.Works best on machine-generated, rather than scanned, PDFs. Built onpdfminer.six.CurrentlytestedonPython 3.7, 3.8, 3.9, 3.10.Translations of this document are available in:Chinese (by @hbh112233abc).To report a bugor request a feature, pleasefile an issue.To ask a questionor request assistance with a specific PDF, pleaseuse the discussions forum.👋 This repository’s maintainers are available to hire for PDF data-extraction consulting projects. To get a cost estimate, contactJeremy(for projects of any size or complexity) and/orSamkit(specifically for table extraction).Table of ContentsInstallationCommand line interfacePython libraryVisual debuggingExtracting tablesExtracting form valuesDemonstrationsComparison to other librariesAcknowledgments / ContributorsContributingInstallationpipinstallpdfplumberCommand line interfaceBasic examplecurl"https://raw.githubusercontent.com/jsvine/pdfplumber/stable/examples/pdfs/background-checks.pdf">background-checks.pdf
pdfplumber<background-checks.pdf>background-checks.csvThe output will be a CSV containing info about every character, line, and rectangle in the PDF.OptionsArgumentDescription--format [format]csvorjson. Thejsonformat returns more information; it includes PDF-level and page-level metadata, plus dictionary-nested attributes.--pages [list of pages]A space-delimited,1-indexed list of pages or hyphenated page ranges. E.g.,1, 11-15, which would return data for pages 1, 11, 12, 13, 14, and 15.--types [list of object types to extract]Choices arechar,rect,line,curve,image,annot, et cetera. Defaults to all available.--laparamsA JSON-formatted string (e.g.,'{"detect_vertical": true}') to pass topdfplumber.open(..., laparams=...).--precision [integer]The number of decimal places to round floating-point numbers. Defaults to no rounding.Python libraryBasic exampleimportpdfplumberwithpdfplumber.open("path/to/file.pdf")aspdf:first_page=pdf.pages[0]print(first_page.chars[0])Loading a PDFTo start working with a PDF, callpdfplumber.open(x), wherexcan be a:path to your PDF filefile object, loaded as bytesfile-like object, loaded as bytesTheopenmethod returns an instance of thepdfplumber.PDFclass.To load a password-protected PDF, pass thepasswordkeyword argument, e.g.,pdfplumber.open("file.pdf", password = "test").To set layout analysis parameters topdfminer.six's layout engine, pass thelaparamskeyword argument, e.g.,pdfplumber.open("file.pdf", laparams = { "line_overlap": 0.7 }).Invalid metadata values are treated as a warning by default. If that is not intended, passstrict_metadata=Trueto theopenmethod andpdfplumber.openwill raise an exception if it is unable to parse the metadata.Thepdfplumber.PDFclassThe top-levelpdfplumber.PDFclass represents a single PDF and has two main properties:PropertyDescription.metadataA dictionary of metadata key/value pairs, drawn from the PDF'sInfotrailers. Typically includes "CreationDate," "ModDate," "Producer," et cetera..pagesA list containing onepdfplumber.Pageinstance per page loaded.Thepdfplumber.PageclassThepdfplumber.Pageclass is at the core ofpdfplumber. Most things you'll do withpdfplumberwill revolve around this class. It has these main properties:PropertyDescription.page_numberThe sequential page number, starting with1for the first page,2for the second, and so on..widthThe page's width..heightThe page's height..objects/.chars/.lines/.rects/.curves/.imagesEach of these properties is a list, and each list contains one dictionary for each such object embedded on the page. For more detail, see "Objects" below.... and these main methods:MethodDescription.crop(bounding_box, relative=False, strict=True)Returns a version of the page cropped to the bounding box, which should be expressed as 4-tuple with the values(x0, top, x1, bottom). Cropped pages retain objects that fall at least partly within the bounding box. If an object falls only partly within the box, its dimensions are sliced to fit the bounding box. Ifrelative=True, the bounding box is calculated as an offset from the top-left of the page's bounding box, rather than an absolute positioning. (SeeIssue #245for a visual example and explanation.) Whenstrict=True(the default), the crop's bounding box must fall entirely within the page's bounding box..within_bbox(bounding_box, relative=False, strict=True)Similar to.crop, but only retains objects that fallentirely withinthe bounding box..outside_bbox(bounding_box, relative=False, strict=True)Similar to.cropand.within_bbox, but only retains objects that fallentirely outsidethe bounding box..filter(test_function)Returns a version of the page with only the.objectsfor whichtest_function(obj)returnsTrue..dedupe_chars(tolerance=1)Returns a version of the page with duplicate chars — those sharing the same text, fontname, size, and positioning (withintolerancex/y) as other characters — removed. (SeeIssue #71to understand the motivation.).extract_text(x_tolerance=3, y_tolerance=3, layout=False, x_density=7.25, y_density=13, **kwargs)Collates all of the page's character objects into a single string.Whenlayout=False: Adds spaces where the difference between thex1of one character and thex0of the next is greater thanx_tolerance. Adds newline characters where the difference between thedoctopof one character and thedoctopof the next is greater thany_tolerance.Whenlayout=True(experimental feature): Attempts to mimic the structural layout of the text on the page(s), usingx_densityandy_densityto determine the minimum number of characters/newlines per "point," the PDF unit of measurement. All remaining**kwargsare passed to.extract_words(...)(see below), the first step in calculating the layout..extract_words(x_tolerance=3, y_tolerance=3, keep_blank_chars=False, use_text_flow=False, horizontal_ltr=True, vertical_ttb=True, extra_attrs=[], split_at_punctuation=False)Returns a list of all word-looking things and their bounding boxes. Words are considered to be sequences of characters where (for "upright" characters) the difference between thex1of one character and thex0of the next is less than or equal tox_toleranceandwhere thedoctopof one character and thedoctopof the next is less than or equal toy_tolerance. A similar approach is taken for non-upright characters, but instead measuring the vertical, rather than horizontal, distances between them. The parametershorizontal_ltrandvertical_ttbindicate whether the words should be read from left-to-right (for horizontal words) / top-to-bottom (for vertical words). Changingkeep_blank_charstoTruewill mean that blank characters are treated as part of a word, not as a space between words. Changinguse_text_flowtoTruewill use the PDF's underlying flow of characters as a guide for ordering and segmenting the words, rather than presorting the characters by x/y position. (This mimics how dragging a cursor highlights text in a PDF; as with that, the order does not always appear to be logical.) Passing a list ofextra_attrs(e.g.,["fontname", "size"]will restrict each words to characters that share exactly the same value for each of thoseattributes, and the resulting word dicts will indicate those attributes. Settingsplit_at_punctuationtoTruewill enforce breaking tokens at punctuations specified bystring.punctuation; or you can specify the list of separating punctuation by pass a string, e.g.,split_at_punctuation='!"&'()*+,.:;<=>?@[]^`{|}~'..search(pattern, regex=True, case=True, **kwargs)Experimental featurethat allows you to search a page's text, returning a list of all instances that match the query. For each instance, the response dictionary object contains the matching text, any regex group matches, the bounding box coordinates, and the char objects themselves.patterncan be a compiled regular expression, an uncompiled regular expression, or a non-regex string. IfregexisFalse, the pattern is treated as a non-regex string. IfcaseisFalse, the search is performed in a case-insensitive manner. The remaining**kwargsare those you would pass to.extract_text(layout=True, ...)..extract_tables(table_settings)Extracts tabular data from the page. For more details see "Extracting tables" below..to_image(**conversion_kwargs)Returns an instance of thePageImageclass. For more details, see "Visual debugging" below. For conversion_kwargs, seehere..close()By default,Pageobjects cache their layout and object information to avoid having to reprocess it. When parsing large PDFs, however, these cached properties can require a lot of memory. You can use this method to flush the cache and release the memory. (In version<= 0.5.25, use.flush_cache().)ObjectsEach instance ofpdfplumber.PDFandpdfplumber.Pageprovides access to several types of PDF objects, all derived frompdfminer.sixPDF parsing. The following properties each return a Python list of the matching objects:.chars, each representing a single text character..lines, each representing a single 1-dimensional line..rects, each representing a single 2-dimensional rectangle..curves, each representing any series of connected points thatpdfminer.sixdoes not recognize as a line or rectangle..images, each representing an image..annots, each representing a single PDF annotation (cf. Section 8.4 of theofficial PDF specificationfor details).hyperlinks, each representing a single PDF annotation of the subtypeLinkand having anURIaction attributeEach object is represented as a simple Pythondict, with the following properties:charpropertiesPropertyDescriptionpage_numberPage number on which this character was found.textE.g., "z", or "Z" or " ".fontnameName of the character's font face.sizeFont size.advEqual to text width * the font size * scaling factor.uprightWhether the character is upright.heightHeight of the character.widthWidth of the character.x0Distance of left side of character from left side of page.x1Distance of right side of character from left side of page.y0Distance of bottom of character from bottom of page.y1Distance of top of character from bottom of page.topDistance of top of character from top of page.bottomDistance of bottom of the character from top of page.doctopDistance of top of character from top of document.matrixThe "current transformation matrix" for this character. (See below for details.)stroking_colorThe color of the character's outline (i.e., stroke), expressed as a tuple or integer, depending on the “color space” used.non_stroking_colorThe character's interior color.object_type"char"Note: A character’smatrixproperty represents the “current transformation matrix,” as described in Section 4.2.2 of thePDF Reference(6th Ed.). The matrix controls the character’s scale, skew, and positional translation. Rotation is a combination of scale and skew, but in most cases can be considered equal to the x-axis skew. Thepdfplumber.ctmsubmodule defines a class,CTM, that assists with these calculations. For instance:frompdfplumber.ctmimportCTMmy_char=pdf.pages[0].chars[3]my_char_ctm=CTM(*my_char["matrix"])my_char_rotation=my_char_ctm.skew_xlinepropertiesPropertyDescriptionpage_numberPage number on which this line was found.heightHeight of line.widthWidth of line.x0Distance of left-side extremity from left side of page.x1Distance of right-side extremity from left side of page.y0Distance of bottom extremity from bottom of page.y1Distance of top extremity bottom of page.topDistance of top of line from top of page.bottomDistance of bottom of the line from top of page.doctopDistance of top of line from top of document.linewidthThickness of line.stroking_colorThe color of the line, expressed as a tuple or integer, depending on the “color space” used.non_stroking_colorThe non-stroking color specified for the line’s path.object_type"line"rectpropertiesPropertyDescriptionpage_numberPage number on which this rectangle was found.heightHeight of rectangle.widthWidth of rectangle.x0Distance of left side of rectangle from left side of page.x1Distance of right side of rectangle from left side of page.y0Distance of bottom of rectangle from bottom of page.y1Distance of top of rectangle from bottom of page.topDistance of top of rectangle from top of page.bottomDistance of bottom of the rectangle from top of page.doctopDistance of top of rectangle from top of document.linewidthThickness of line.stroking_colorThe color of the rectangle's outline, expressed as a tuple or integer, depending on the “color space” used.non_stroking_colorThe rectangle’s fill color.object_type"rect"curvepropertiesPropertyDescriptionpage_numberPage number on which this curve was found.pointsPoints — as a list of(x, top)tuples — describing the curve.heightHeight of curve's bounding box.widthWidth of curve's bounding box.x0Distance of curve's left-most point from left side of page.x1Distance of curve's right-most point from left side of the page.y0Distance of curve's lowest point from bottom of page.y1Distance of curve's highest point from bottom of page.topDistance of curve's highest point from top of page.bottomDistance of curve's lowest point from top of page.doctopDistance of curve's highest point from top of document.linewidthThickness of line.fillWhether the shape defined by the curve's path is filled.stroking_colorThe color of the curve's outline, expressed as a tuple or integer, depending on the “color space” used.non_stroking_colorThe curve’s fill color.object_type"curve"Additionally, bothpdfplumber.PDFandpdfplumber.Pageprovide access to two derived lists of objects:.rect_edges(which decomposes each rectangle into its four lines) and.edges(which combines.rect_edgeswith.lines).imageproperties[To be completed.]Obtaining higher-level layout objects viapdfminer.sixIf you pass thepdfminer.six-handlinglaparamsparameter topdfplumber.open(...), then each page's.objectsdictionary will also containpdfminer.six's higher-level layout objects, such as"textboxhorizontal".Visual debuggingNote:To usepdfplumber's visual-debugging tools, you'll also need to have two additional pieces of software installed on your computer:ImageMagick.Installation instructions here.ghostscript.Installation instructions here, or simplyapt install ghostscript(Ubuntu) /brew install ghostscript(Mac).Creating aPageImagewith.to_image()To turn any page (including cropped pages) into anPageImageobject, callmy_page.to_image(). You can optionally pass aresolution={integer}keyword argument, which defaults to 72. E.g.:im=my_pdf.pages[0].to_image(resolution=150)From a script or REPL,im.show()will open the image in your local image viewer. ButPageImageobjects also play nicely with IPython/Jupyter notebooks; they automatically render as cell outputs. For example:Note:.to_image(...)works as expected withPage.crop(...)/CroppedPageinstances, but is unable to incorporate changes made viaPage.filter(...)/FilteredPageinstances.BasicPageImagemethodsMethodDescriptionim.reset()Clears anything you've drawn so far.im.copy()Copies the image to a newPageImageobject.im.show()Opens the image in your local image viewer.im.save(path_or_fileobject, format="PNG")Saves the annotated image.Drawing methodsYou can pass explicit coordinates or anypdfplumberPDF object (e.g., char, line, rect) to these methods.Single-object methodBulk methodDescriptionim.draw_line(line, stroke={color}, stroke_width=1)im.draw_lines(list_of_lines, **kwargs)Draws a line from aline,curve, or a 2-tuple of 2-tuples (e.g.,((x, y), (x, y))).im.draw_vline(location, stroke={color}, stroke_width=1)im.draw_vlines(list_of_locations, **kwargs)Draws a vertical line at the x-coordinate indicated bylocation.im.draw_hline(location, stroke={color}, stroke_width=1)im.draw_hlines(list_of_locations, **kwargs)Draws a horizontal line at the y-coordinate indicated bylocation.im.draw_rect(bbox_or_obj, fill={color}, stroke={color}, stroke_width=1)im.draw_rects(list_of_rects, **kwargs)Draws a rectangle from arect,char, etc., or 4-tuple bounding box.im.draw_circle(center_or_obj, radius=5, fill={color}, stroke={color})im.draw_circles(list_of_circles, **kwargs)Draws a circle at(x, y)coordinate or at the center of achar,rect, etc.Note: The methods above are built on Pillow'sImageDrawmethods, but the parameters have been tweaked for consistency with SVG'sfill/stroke/stroke_widthnomenclature.Troubleshooting ImageMagick on Debian-based systemsIf you're usingpdfplumberon a Debian-based system and encounter aPolicyError, you may be able to fix it by changing the following line in/etc/ImageMagick-6/policy.xmlfrom this:<policydomain="coder"rights="none"pattern="PDF"/>... to this:<policydomain="coder"rights="read|write"pattern="PDF"/>(More details aboutpolicy.xmlavailable here.)Extracting tablespdfplumber's approach to table detection borrows heavily fromAnssi Nurminen's master's thesis, and is inspired byTabula. It works like this:For any given PDF page, find the lines that are (a) explicitly defined and/or (b) implied by the alignment of words on the page.Merge overlapping, or nearly-overlapping, lines.Find the intersections of all those lines.Find the most granular set of rectangles (i.e., cells) that use these intersections as their vertices.Group contiguous cells into tables.Table-extraction methodspdfplumber.Pageobjects can call the following table methods:MethodDescription.find_tables(table_settings={})Returns a list ofTableobjects. TheTableobject provides access to the.cells,.rows, and.bboxproperties, as well as the.extract(x_tolerance=3, y_tolerance=3)method..extract_tables(table_settings={})Returns the text extracted fromalltables found on the page, represented as a list of lists of lists, with the structuretable -> row -> cell..extract_table(table_settings={})Returns the text extracted from thelargesttable on the page, represented as a list of lists, with the structurerow -> cell. (If multiple tables have the same size — as measured by the number of cells — this method returns the table closest to the top of the page.).debug_tablefinder(table_settings={})Returns an instance of theTableFinderclass, with access to the.edges,.intersections,.cells, and.tablesproperties.For example:pdf=pdfplumber.open("path/to/my.pdf")page=pdf.pages[0]page.extract_table()Click here for a more detailed example.Table-extraction settingsBy default,extract_tablesuses the page's vertical and horizontal lines (or rectangle edges) as cell-separators. But the method is highly customizable via thetable_settingsargument. The possible settings, and their defaults:{"vertical_strategy":"lines","horizontal_strategy":"lines","explicit_vertical_lines":[],"explicit_horizontal_lines":[],"snap_tolerance":3,"snap_x_tolerance":3,"snap_y_tolerance":3,"join_tolerance":3,"join_x_tolerance":3,"join_y_tolerance":3,"edge_min_length":3,"min_words_vertical":3,"min_words_horizontal":1,"keep_blank_chars":False,"text_tolerance":3,"text_x_tolerance":3,"text_y_tolerance":3,"intersection_tolerance":3,"intersection_x_tolerance":3,"intersection_y_tolerance":3,}SettingDescription"vertical_strategy"Either"lines","lines_strict","text", or"explicit". See explanation below."horizontal_strategy"Either"lines","lines_strict","text", or"explicit". See explanation below."explicit_vertical_lines"A list of vertical lines that explicitly demarcate cells in the table. Can be used in combination with any of the strategies above. Items in the list should be either numbers — indicating thexcoordinate of a line the full height of the page — orline/rect/curveobjects."explicit_horizontal_lines"A list of horizontal lines that explicitly demarcate cells in the table. Can be used in combination with any of the strategies above. Items in the list should be either numbers — indicating theycoordinate of a line the full height of the page — orline/rect/curveobjects."snap_tolerance","snap_x_tolerance","snap_y_tolerance"Parallel lines withinsnap_tolerancepixels will be "snapped" to the same horizontal or vertical position."join_tolerance","join_x_tolerance","join_y_tolerance"Line segments on the same infinite line, and whose ends are withinjoin_toleranceof one another, will be "joined" into a single line segment."edge_min_length"Edges shorter thanedge_min_lengthwill be discarded before attempting to reconstruct the table."min_words_vertical"When using"vertical_strategy": "text", at leastmin_words_verticalwords must share the same alignment."min_words_horizontal"When using"horizontal_strategy": "text", at leastmin_words_horizontalwords must share the same alignment."keep_blank_chars"When using thetextstrategy, consider" "chars to bepartsof words and not word-separators."text_tolerance","text_x_tolerance","text_y_tolerance"When thetextstrategy searches for words, it will expect the individual letters in each word to be no more thantext_tolerancepixels apart."intersection_tolerance","intersection_x_tolerance","intersection_y_tolerance"When combining edges into cells, orthogonal edges must be withinintersection_tolerancepixels to be considered intersecting.Table-extraction strategiesBothvertical_strategyandhorizontal_strategyaccept the following options:StrategyDescription"lines"Use the page's graphical lines — including the sides of rectangle objects — as the borders of potential table-cells."lines_strict"Use the page's graphical lines — butnotthe sides of rectangle objects — as the borders of potential table-cells."text"Forvertical_strategy: Deduce the (imaginary) lines that connect the left, right, or center of words on the page, and use those lines as the borders of potential table-cells. Forhorizontal_strategy, the same but using the tops of words."explicit"Only use the lines explicitly defined inexplicit_vertical_lines/explicit_horizontal_lines.NotesOften it's helpful to crop a page —Page.crop(bounding_box)— before trying to extract the table.Table extraction forpdfplumberwas radically redesigned forv0.5.0, and introduced breaking changes.Extracting form valuesSometimes PDF files can contain forms that include inputs that people can fill out and save. While values in form fields appear like other text in a PDF file, form data is handled differently. If you want the gory details, see page 671 of thisspecification.pdfplumberdoesn't have an interface for working with form data, but you can access it usingpdfplumber's wrappers aroundpdfminer.For example, this snippet will retrieve form field names and values and store them in a dictionary. You may have to modify this script to handle cases like nested fields (see page 676 of the specification).pdf=pdfplumber.open("document_with_form.pdf")fields=pdf.doc.catalog["AcroForm"].resolve()["Fields"]form_data={}forfieldinfields:field_name=field.resolve()["T"]field_value=field.resolve()["V"]form_data[field_name]=field_valueDemonstrationsUsingextract_tableon a California Worker Adjustment and Retraining Notification (WARN) report. Demonstrates basic visual debugging and table extraction.Usingextract_tableon the FBI's National Instant Criminal Background Check System PDFs. Demonstrates how to use visual debugging to find optimal table extraction settings. Also demonstratesPage.crop(...)andPage.extract_text(...).Inspecting and visualizingcurveobjects.Extracting fixed-width data from a San Jose PD firearm search report, an example of usingPage.extract_text(...).Comparison to other librariesSeveral other Python libraries help users to extract information from PDFs. As a broad overview,pdfplumberdistinguishes itself from other PDF processing libraries by combining these features:Easy access to detailed information about each PDF objectHigher-level, customizable methods for extracting text and tablesTightly integrated visual debuggingOther useful utility functions, such as filtering objects via a crop-boxIt's also helpful to know what featurespdfplumberdoesnotprovide:PDFgenerationPDFmodificationOptical character recognition (OCR)Strong support for extracting tables from OCR'ed documentsSpecific comparisonspdfminer.sixprovides the foundation forpdfplumber. It primarily focuses on parsing PDFs, analyzing PDF layouts and object positioning, and extracting text. It does not provide tools for table extraction or visual debugging.PyPDF2is a pure-Python library "capable of splitting, merging, cropping, and transforming the pages of PDF files. It can also add custom data, viewing options, and passwords to PDF files." It can extract page text, but does not provide easy access to shape objects (rectangles, lines, etc.), table-extraction, or visually debugging tools.pymupdfis substantially faster thanpdfminer.six(and thus alsopdfplumber) and can generate and modify PDFs, but the library requires installation of non-Python software (MuPDF). It also does not enable easy access to shape objects (rectangles, lines, etc.), and does not provide table-extraction or visual debugging tools.camelot,tabula-py, andpdftablesall focus primarily on extracting tables. In some cases, they may be better suited to the particular tables you are trying to extract.Acknowledgments / ContributorsMany thanks to the following users who've contributed ideas, features, and fixes:Jacob FentonDan NguyenJeff BarreraBob LannonDustin Tindall@yevgnen@meldonizationOisín MoranSamkit JainFrancisco ArandaKwok-kuen CheungMarcoIdan David@xv44586Alexander RegueiroDaniel Peña@bobluda@ramcdona@johnhugeJhonatan LopesEthan CoreyShannon ShenContributingPull requests are welcome, but please submit a proposal issue first, as the library is in active development.Current maintainers:Jeremy Singer-VineSamkit Jain