Package: similarity-processor
Text SimilarityTool to identify the similarity of the input textIt can be used to identify the similarity of,TestsCodeRequirementsDefectsAdvantage of using such similarity analysis are,Resolving technical debtGrouping together similar code / tests / requirements / defects etc.Dependenciespython 3.8 : 64 bitpython packages (xlrd, xlsxwriter, pandas, scikit-learn, numpy)InstallationRequirements are added in requirement.txt filePython 3.8python:
install python for the respective OS athttps://www.python.org/downloads/Make sure to update the path variable to point to the python installation folder.pip: (only if pip is not present by default)
get get-pip.py from below link to your folderhttps://bootstrap.pypa.io/get-pip.pyOpen a command prompt and navigate to the folder containingget-pip.py. Run the
following command:similarity-processor:pipinstallsimilarity-processorpipinstallsimilarity-processorUsageUI>>>python-msimilarity.similarity_uiPath to the test/requirement/other other document to be
analyzed(xlsx / csv format).Unique ID in the csv/xlsx column ID(0/1 etc...)Steps/Description id for content matching (column of interest IDs
in the csv/xlsx separated by , like 1,2,3)If new requirement / test to me checked with existing, enable the
check box and paste the content to be checked in the new text box.Commandline>>>python-msimilarity--p"path\to\TestBank.xlsx"--u0--c"1,2,3"--n8Help option can be found at,>>>python-msimilarity--hCode>>>fromsimilarity.similarity_ioimportSimilarityIO
>>>similarity_io_obj=SimilarityIO("path\to\TestBank.xlsx",0,"1,2,3")>>>similarity_io_obj.orchestrate_similarity()ArgumentsMandatoryPath to the input fileUnique id value column id in xlsxInterested columns in xlsxOptionalUpper and lower range to filter the similarity values in the output
(defaulted "60,100")Number of rows in the html report, defaulted to 100Are you checking a new text against a existing text bank?If yes: new textFilter value to split the report xlsx file, defaulted to 500000,
500001 onward row will be moved to new fileimportpandasaspd
fromsimilarity.similarity_ioimportSimilarityIOdemo_df=pd.read_excel(r"input\xlsx\sheet\name")# You could read from any input sourcesimilarity_io_obj=SimilarityIO(None,None,None)# (None, None, None, 200) =>200 = The brief html report rowsdefaultis10similarity_io_obj.file_path=r"path\to\report\folder"#when used in this format, else input file path to read datasimilarity_io_obj.data_frame=demo_df# input data framesimilarity_io_obj.uniq_header="Uniq ID"# Unique header of the input data frame (string)similarity_io_obj.create_merged_df()processed_similarity=similarity_io_obj.process_cos_match()similarity_io_obj.report_brief_html(processed_similarity)processed_similarity.to_csv(r"path\to\report\folder\report.csv",header=True)OutputOutput will be available in same folder as input file orfile_pathspecifiedIf any duplicate ids in the unique id file with name string containing
'duplicate id'A recommendation file with similarity valuesA merged file with data in the "interested columns in xlsx"An html brief report containing the top 10 similarities
(100 is default value which can be changed by --n option)ContactBrijesh Krishnanbrijesh.krishnank@philips.comDattatreya Vellaldsvellal@philips.comLicenseThe MIT License (MIT) Copyright © [2019] Koninklijke Philips N.V,https://www.philips.comPermission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the “Software”),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.