Package: pic2vec
Pic2VecFeaturize images using a small, contained pre-trained deep learning networkFree software: BSD licenseFeaturesThis is the prototype for image features engineering. Supports Python 2.7, 3.4, 3.5, 3.6, and 3.7pic2vecis a python package that performs automated feature extraction
for image data. It supports feature engineering on new image data, and allows
traditional machine learning algorithms (such as tree-based algorithms) to
train on image data.Input SpecificationData Formatpic2vecworks on image data represented as either:A directory of image files.As URL pointers contained in a CSV.Or as a directory of images with a CSV containing pointers to the image files.If no CSV is provided with the directory, it automatically generates a CSV to store the features with the appropriate images.Each row of the CSV represents a different image, and image rows can also have columns containing other data about the images as well. Each image's featurized representation will be appended as a series of new columns at the end of the appropriate image row.Constraints SpecificationThe goal of this project was to make the featurizer as easy to use and hard to break as possible. If working properly, it should be resistant to badly-formatted data, such as missing rows or columns in the csv, image mismatches between a CSV and an image directory, and invalid image formats.However, for the featurizer to function optimally, it prefers certain constraints:The CSV should have no missing columns or rows, and there should be full overlap between images in the CSV and the image directoryIf checking predictions on a separate test set (such as on Kaggle), the filesystem needs to sort filepaths consistently with the sorting of the test set labels. The order in the CSV (whether generated automatically or passed in) will be considered the canonical order for the feature vectors.The featurizer can only process .png, .jpeg, or .bmp image files. Any other images will be left out of the featurization by being represented by zero vectors in the image batch.Quick StartThe following Python code shows a typical usage ofpic2vec:frompic2vecimportImageFeaturizerimage_column_name='images'my_csv='path/to/data.csv'my_image_directory='path/to/image/directory/'my_featurizer=ImageFeaturizer(model='xception',depth=2,autosample=True)featurized_df=my_featurizer.featurize(image_column_name,csv_path=my_csv,image_path=my_image_directory)ExamplesTo get started, see the following example:Cats vs. Dogs: Dataset from combined directory + CSVExamples coming soon:
2. Hot Dog, Not Hot Dog: Dataset from a CSV with URLs and no image directoryInstallationSee theInstallation Guidefor details.Installing Keras/TensorflowIf you run into trouble installing Keras or Tensorflow as a dependency, read theKeras installation guideandTensorflow installation guidefor details about installing Keras/Tensorflow on your machine.Using Featurizer Output With DataRobotpic2vecgenerates a flat CSV which is ready for supervised modeling, if the data has been labelled with a variable that
can be used as a target. The images are transformed into a set of regular columns containing numeric data.
Additionally, if unlabelled, it can be used for unsupervised learning (such as anomaly detection).Running testsTo run the unit tests withpytest, runpy.test testsCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.=======
History0.101.1 (2019-9-25)Limited Keras version to pre-2.3.0 to fix issues from Keras' breaking changes0.101.0 (2019-3-25)Updated version of Trafaret to a non-beta versionUpdated keras to 2.2.3 or greaterThis library upgrade changes prediction consistency across past versions of pic2vec.
ResNet50 is the model that has changed the most, due to changes in implementation.
Other models have small floating point changes, but still pass np.testing.assert_allclose tests.0.100.1 (2019-3-24)Updated version of Pillow to 5.4.1, in order to support Python 3.7Updated the README0.100.0 (2018-12-10)Added test coverage and increased error checkingChanged default csv nameChangedimage_column_headerstoimage_columnseverywhereUpdated examplesUpdated version of scipy to 1.1 and numpy to 1.150.99.2 (2018-08-01)Updated the notebook exampleSome code cleanup0.99.1 (2018-06-20)Lots of code cleanupChanged new_csv_name argument to new_csv_path everywhere for consistencyRemoved '_full' from the saved csv_name for the full dataframe. Features-only csv still has
'_features_only' in csv name.Added 'featurized' to saved csv namesRemoved new_csv_path as argument to functions that do not actually require it0.99.0 (2018-04-02)Added batch processingMade pic2vec more programmatic (removed automatic csv-writing, etc.)Bound keras to <2.1.5 to remove resnet problem0.9.0 (2017-09-24)Fixed Keras backwards compatibility issues (include_top deprecated, require_flatten added)Fixed ResNet50 update issues (removed a zero-padding layer, updated weights)0.8.2 (2017-08-14)Updated trafaret requirement for PyPi packageUpdated cats vs. dogs example0.8.1 (2017-08-07)Fixed bugs with robust namingAdded error message for failed image conversion0.8.0 (2017-08-02)Added robust naming options to the generated csv files0.7.1 (2017-08-02)Fixed PIL truncated image bug0.7.0 (2017-08-02)Fixed bug with CSV badly formed URLsFixed mistake with InceptionV3 preprocessing happening for every model0.6.3 (2017-07-25)Added Travis and Coveralls for testing and coverage automationRepo went publicPython 3.x compatibility0.6.2 (2017-07-14)Fixed image format recognition.0.6.1 (2017-07-12)Directory-only now natural sorted.0.6.0 (2017-07-11)Added multi-column supportAdded missing image column to csv0.5.0 (2017-07-06)Renamed to pic2vecTests parametrized0.4.3 (2017-07-03)Second round of code review- optimized code, better type checking with trafaret0.4.2 (2017-06-30)Improved README test examples0.4.1 (2017-06-30)Fixed documentation0.4.0 (2017-06-29)Added ability to call multiple models, and packaged in SqueezeNet with weights.0.3.0 (2017-06-26)Created installation instructions and readme files, ready for prototype distribution0.2.9(2017-06-25)Fixed import problem that prevented generated csvs from saving0.2.8(2017-06-25)Fixed variable name bugs0.2.7(2017-06-25)Changed image_directory_path to the more manageable image_pathMade testing module and preprocessing module slightly more robust.0.2.6(2017-06-23)Added features-only csv test, and got rid of the column headers in the fileAdded Documentation to data featurization modeules0.2.5(2017-06-23)100% test coverageFixed a problem where a combined directory + csv was appending to the wrong
rows when there was a mismatch between the directory and the csv.0.2.4(2017-06-22)Fixed more bugs in build_featurizer0.2.3(2017-06-22)Fixed build_featurizer troubles with building new csv paths in current directory0.2.2(2017-06-22)Full requirements for keras imported0.2.1 (2017-06-22)Bug fixes0.2.0 (2017-06-22)Second release on PyPI.Install keras with tensorflow backend specifically0.1.0 (2017-06-14)First release on PyPI.