Package: data-harvest-reader
data-harvest-readerFeaturesReading Various File FormatsDirectory and ZIP File HandlingData JoiningDeduplicationCustom FiltersLoggingInstallation RequirementspipinstallpolarsloguruUsageInitializationfromdata_harvest_readerimportDataReaderdata_reader=DataReader(log_to_file=True,log_file="data_reader.log")Reading DataFrom Directorydata=data_reader.read_data('path/to/directory',join_similar=True)From ZIP Filedata=data_reader.read_data('path/to/zipfile.zip',join_similar=False)Applying Deduplicationduplicated_subset_dict={'file1':['column1','column2']}data=data_reader.read_data('path/to/source',duplicated_subset_dict=duplicated_subset_dict)Applying Filtersfilter_subset={'file1':[{'column':'Col1','operation':'>','values':100},{'column':'Col2','operation':'==','values':'Value'}]}data=data_reader.read_data('path/to/source',filter_subset=filter_subset)Handling Exceptionstry:data=data_reader.read_data('path/to/source')exceptUnsupportedFormatError:print("Unsupported file format provided")exceptFilterConfigurationError:print("Error in filter configuration")Exampledata_reader=DataReader()data=data_reader.read_data(r'C:\path\to\data',join_similar=True,filter_subset={'example_file':[{'column':'Age','operation':'>','values':30}]})Contributing to DataReaderGetting StartedFork the Repository: Start by forking the main repository. This creates your own copy of the project where you can make changes.Clone the Forked Repository: Clone your fork to your local machine. This step allows you to work on the codebase directly.Set Up the Development Environment: Ensure you have all necessary dependencies installed. It's recommended to use a virtual environment.Create a New Branch: Always create a new branch for your changes. This keeps the main branch stable and makes reviewing changes easier.Making ContributionsMake Your Changes: Implement your feature, fix a bug, or make your proposed changes. Ensure your code adheres to the project's coding standards and guidelines.Test Your Changes: Before submitting, test your changes thoroughly. Write unit tests if applicable, and ensure all existing tests pass.Document Your Changes: Update the documentation to reflect your changes. If you're adding a new feature, include usage examples.
Push your changes to your fork on GitHub.Commit Your Changes: Make concise and clear commit messages, describing what each commit does.Push to Your Fork: Push your changes to your fork on GitHub.Create a Pull Request (PR): Go to the originalDataReaderrepository and create a pull request from your fork. Ensure you describe your changes in detail and link any relevant issues.Review ProcessAfter submitting your PR, the maintainers will review your changes. Be responsive to feedback:Respond to Comments: If the reviewers ask for changes, make them promptly. Discuss any suggestions or concerns.Update Your PR: If needed, update your PR based on feedback. This may involve adding more tests or tweaking your approach.Final StepsOnce your PR is approved:Merge: The maintainers will merge your changes into the main codebase.Stay Engaged: Continue to stay involved in the project. Look out for feedback from users on your new feature or fix.ConclusionContributing toDataReaderis a rewarding experience that benefits the entire user community. Your contributions help makeDataReadera more robust and versatile tool. We welcome developers of all skill levels and appreciate every form of contribution, from code to documentation. Thank you for considering contributing toDataReader!