Package: unit-parse
Unit Parse (unit_parse)Do you have strings/text that you want to turn into quantities?Are you trying to clean scientific data you extracted fromWikipidaor some
other sketchy website?Try 'Unit_Parse' to clean everything up for you!Description:'Unit_Parse' is built on top ofPint. It was specifically designed to handle data
that was extracted from scientific work. It has been rigorously tested against chemistry data
extracted from Wikipida (example:styrene; density, melting point, boiling
point, etc.) and data fromPubChem(example:styrene; density, melting point, flash point, etc.).Installationpip install unit_parseDependenciesPint- Provides unit conversions of cleaned and parsed quantities.UsageBasicsPass string you want to parse toparser().fromunit_parseimportparserresult=parser("1.23 g/cm3 (at 25 °C)")print(result)# [[<Quantity(1.23, 'g / cm ** 3')>, <Quantity(25, 'degC')>]]'Quantity' arepint quantities.Output structureParse unsuccessful: NoneSingle value:quantity5 g/molSingle value with condition:[[quantity, condition]][['25 degC', '1 bar']]boil temperature is 25 °C at 1 bar of pressureMultiple values with conditions:[[quantity, condition], [quantity, condition], ...][['25 degC', '1 bar'], ['50 degC', '5 bar'], ['100 degC', '10 bar']]Merging QuantitiesSometimes when you are doing parsing, you get multiple values from the parser. So it would be nice to reduce it
down just to one value/value+condition/series+condition.reduce_quantitiesdoes exactly that!It will group approximate equivalent quantities and throw out bad units (units that are not like the most common).
You can select your preference for return priority with theorderparameter.fromunit_parseimportQuantity,reduce_quantitiesquantities=[Quantity("68 degF"),Quantity("68.0 degF"),Quantity("20.0 degC"),Quantity("293.15 kelvin * speed_of_light ** 2")]result=reduce_quantities(quantities)print(result)# Quantity("68 degF")LoggingThe logger can be used to track the parsing steps.Default level is warning.warning: will only let you know if there is any text that is being ignored in the parsing process.
info: will show the major parsing steps.
debug: will show fine grain parsing steps.Example: INFOCode:importloggingfromunit_parseimportparser,loggerlogger.setLevel(logging.INFO)result=parser("37.34 kJ/mole (at 25 °C)")print(result)Output:INPUT: 37.34 kJ/mole (at 25 °C)substitution: ('37.34 kJ/mole (at 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)multiple_quantities_main: ('37.34 kJ/mole ( @ 25 °C)',) --> [['37.34 kJ/mole', '', '25 °C']]text_list_to_quantity: ([['37.34 kJ/mole', '', '25 °C']],) --> [[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]]remove_duplicates: ([[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]],) --> [[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]]OUTPUT: [<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>][<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]Example: DEBUGCode:importloggingfromunit_parseimportparser,loggerlogger.setLevel(logging.DEBUG)result=parser("37.34 kJ/mole (at 25 °C)")print(result)Output:INPUT: 37.34 kJ/mole (at 25 °C)sub_general: ('37.34 kJ/mole (at 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)sub_power: ('37.34 kJ/mole ( @ 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)sub_sci_notation: ('37.34 kJ/mole ( @ 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)reduce_ranges: ('37.34 kJ/mole ( @ 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)substitution: ('37.34 kJ/mole (at 25 °C)',) --> 37.34 kJ/mole ( @ 25 °C)multiple_quantities: ('37.34 kJ/mole ( @ 25 °C)',) --> ['37.34 kJ/mole ( @ 25 °C)']reduce_parenthesis: ('37.34 kJ/mole ( @ 25 °C)',) --> ['37.34 kJ/mole ', ' @ 25 °C']condition_finder: ('37.34 kJ/mole ( @ 25 °C)',) --> ['37.34 kJ/mole', '', '25 °C']multiple_quantities_main: ('37.34 kJ/mole ( @ 25 °C)',) --> [['37.34 kJ/mole', '', '25 °C']]get_quantity_and_cond: (['37.34 kJ/mole', '', '25 °C'],) --> [<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]text_list_to_quantity: ([['37.34 kJ/mole', '', '25 °C']],) --> [[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]]remove_duplicates: ([[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]],) --> [[<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]]OUTPUT: [<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>][<Quantity(37.34, 'kilojoule / mole')>, <Quantity(25, 'degree_Celsius')>]ExamplesYep, there's alot of them!# Simple conversions5-->55g-->5g5g/ml-->5.0g/ml1K-->1K40°F-->40°F-40°F-->-40°F170°C-->170°C40°F-->40°F20.80mmHg-->20.8mmHg20.80mmHg-->20.8mmHg# scientific notation15*10**2s-->1500s15*10^2s-->1500s1510**2s-->1500s8.20x10**+1ppm-->82.0ppm8.20x10+1ppm-->82.0ppm5e1g/mol-->50.0g/mol5E1g/mol-->50.0g/mol5e1g/mol-->50.0g/mol5E1g/mol-->50.0g/mol5e+1g/mol-->50.0g/mol5E-1g/mol-->0.5g/mol−66.11·10-62ml/mol-->-6.611e-61ml/mol−66.11·10+62ml/mol-->-6.611e+63ml/mol−66.11·1062ml/mol-->-6.611e+63ml/mol# messed up units/ units with powers2.3gcm-->2.3cm*g5e5gmol/s-->500000.0g*mol/s2.3gcm**3-->2.3cm**3*g2.3gcm**3-->2.3cm**3*g2.3g-->2.3g1.10*10**-05atm-m**3/mole-->1.1000000000000001e-05atm*m**3/mol-54.6e-5atm-m**3/mole-->-0.000546atm*m**3/mol2.3mlgcm-->2.3cm*g*ml42.3gcm-3-->42.3g/cm**342.3gcm-3-->42.3g/cm**3−66.11·10-62cm3/mol-->-6.611e-61cm**3/mol−66.11·10+62cm3/mol-->-6.611000000000001e+63cm**3/mol−66.11·1062cm3/mol-->-6.611000000000001e+63cm**3/mol345.234KCAL/MOLE-->345.234kcal/mol# parenthesis (brackets turn into parenthesis)(4.0°C)-->4.0°C[4.0°C]-->4.0°C4.0(°C)-->4.0°C4.0(°C))-->4.0°C)4.0(°C)-->4.0°C(4.0(°C)-->4.0°C()4.0(°C)-->4.0°C4.0°C[39.2g/[mol*s]]-->[[<Quantity(4.0,'degree_Celsius')>,<Quantity(39.2,'gram / mole / second')>]]1.0722at68°F(EPA,1998)-->[[1.0722,<Quantity(68,'degree_Fahrenheit')>]]# conditions37.34kJ/mole(at25°C)-->[[<Quantity(37.34,'kilojoule / mole')>,<Quantity(25,'degree_Celsius')>]]20.8mmHg@25°C-->[[<Quantity(20.8,'millimeter_Hg')>,<Quantity(25,'degree_Celsius')>]]20.8mmHg(25°C)-->[[<Quantity(20.8,'millimeter_Hg')>,<Quantity(25,'degree_Celsius')>]]20.8mmHgat25°C-->[[<Quantity(20.8,'millimeter_Hg')>,<Quantity(25,'degree_Celsius')>]]-4,395.63kJ/molat25°C-->[[<Quantity(-4395.63,'kilojoule / mole')>,<Quantity(25,'degree_Celsius')>]]# list of quantities18mmHg;20mmHg-->20mmHg18mmHg@68°F;20mmHg@77°F-->[[<Quantity(18,'millimeter_Hg')>,<Quantity(68,'degree_Fahrenheit')>],[<Quantity(20,'millimeter_Hg')>,<Quantity(77,'degree_Fahrenheit')>]]18mmHg@68°F;20mmHg@77°F(NTP,1992)-->[[<Quantity(18,'millimeter_Hg')>,<Quantity(68,'degree_Fahrenheit')>],[<Quantity(20,'millimeter_Hg')>,<Quantity(77,'degree_Fahrenheit')>]]18mmHgat68°F;20mmHgat77°F-->[[<Quantity(18,'millimeter_Hg')>,<Quantity(68,'degree_Fahrenheit')>],[<Quantity(20,'millimeter_Hg')>,<Quantity(77,'degree_Fahrenheit')>]]Lowthreshold=13.1150mg/cum;Highthreshold=26840mg/cum;Irritatingconcn=22875mg/cum.-->22875.0mg/m**3# ranges115.2-115.3°C-->115.2°C115.2-115.3°C-->115.2°C# words8.20x10+1ppm;pure-->82.0ppm40°F(NTP,1992)-->40°F4.0°C(39.2°F)-closedcup-->4.0°C4.0°C[39.2g/[mol*s]]-closedcup-->[[<Quantity(4.0,'degree_Celsius')>,<Quantity(39.2,'gram / mole / second')>]]4.0°C[39.2g/[mol*s]approx.]-closedcup-->[[<Quantity(4.0,'degree_Celsius')>,<Quantity(39.2,'gram / mole / second')>]]4.0°C[39.2g/[mol*s]approx.]-closedcup-->[[<Quantity(4.0,'degree_Celsius')>,<Quantity(39.2,'gram / mole / second')>]]4.0°C[39.2g/[mol*s]approx.]-closedcup-->[[<Quantity(4.0,'degree_Celsius')>,<Quantity(39.2,'gram / mole / second')>]]Detectioninwater:0.73ppm;Chemicallypure-->0.73ppmOdorThresholdRange:0.15to25ppm-->0.15ppm0.05ppmpurityspecified-->0.05ppmOdordetectioninair,0.05ppm(puritynotspecified)-->0.05ppmRelativedensity(water=1):1.04-1.13-->1.04Densityapproximately6.5lb/gal.-->6.5lb/gal# duplicates of same quantity different units4.0°C(39.2°F)-->4.0°C-7991cal/g=-334.6X10+5J/KG-->-33460000.000000004J/kg# complex18mmHgat68°F;20mmHgat77°F(NTP,1992)-->[[<Quantity(18,'millimeter_Hg')>,<Quantity(68,'degree_Fahrenheit')>],[<Quantity(20,'millimeter_Hg')>,<Quantity(77,'degree_Fahrenheit')>]]Soundtravelsat0.34km/s-->0.34km/sPassmea300mlbeer.-->300mlStuff it gets wrong. No one is perfect!Indexofrefraction:1.50920@20°C/D-->[[1.5092,<Quantity(293.15,'kelvin / debye')>]]Vaporpressure,kPaat20°C:2.0-->2.0Specificopticalrotation@15°C/D+230deg(concnbyvolume=1.8inchloroform)-->1.8ConfigurationThe parser has a few configurations exposed to make it easy to modify how the works.Remove wordsText you want removed prior to parsing.Default is None. (Note: the parser naturally takes care of alot of 'bad' text)importunit_parseremove_words=["approx.","roughly","close to"]unit_parse.config.remove_text=remove_wordsresult=unit_parse.parser("approx. 100 grams")print(result)# Quantity("100 gram")Pre-Processing SubstitutionsText you want to replace with another.Default there is a big list. Regex or text is accepted.Defaults:pre_proc_sub=[# [pattern, substitution value]["^[a-zA-Z;,.: /]*",""],# remove text at front of strings["(?<=[^a-zA-Z])at([^a-zA-Z])"," @ "],# replace at with @["−","-"],# unify dash (long, short) symbols["·","*"],# unify multiplication symbols["° F"," °F"],# pint gets confused (degree farad)["° C"," °C"],# pint gets confused["°F","degF"],# eliminates issue with capitalization step["°C","degC"],# eliminates issue with capitalization step["(?<=[0-9]{1})[ ]{0,1}X[ ]{0,1}(?=[0-9]{1})","*"],# unify multiplication symbols["(?<=[0-9]{1})[ ]{0,1}x[ ]{0,1}(?=[0-9]{1})","*"],# unify multiplication symbols["\[","("],# make all brackets parenthesis["\]",")"],# make all brackets parenthesis["^.*={1}",""],# delete everything in front of equal["^.*:{1}",""],# delete everything in front of collen["( to )","-"],# unify how range are represented["(?<=[a-zA-Z])-(?=[a-zA-Z])"," "],# turn dashes between text into spaces so dictionary can remove["mm Hg","mmHg"],# pint gets confused["KG","kg"],# pint gets confused["LB","lb"],# pint gets confused["kpa","kPa"],# pint gets confused["cu ft","ft**3"],# pint gets confused["cu in","in**3"],# pint gets confused["cu m","m**3"],# pint gets confused["cu cm","cm**3"],# pint gets confused["cu mm","mm**3"],# pint gets confused]importunit_parsemore_pre_processing=[["MOL","mol"]]# [bad text/regex, new text]unit_parse.config.pre_proc_sub+=more_pre_processing# Here we are adding to the existing listresult=unit_parse.parser("100 MOL")# pint is case-sensitive, so this will result in an invalid unitprint(result)# Quantity("100 mole")Last Minute SubstitutionsText you want to replace with another, but happens at the very last stage before trying to convert to a unit. You
should try to use pre-substitutions first, but there can be some situations where you want to do substitutions at a
semi-parsed quantity.Default there is a big list. Regex or text is accepted.Defaults:last_minute_sub=[# [pattern, substitution value]["-{1}[^0-9]*$",""],# remove trailing dash["(?<=[a-zA-Z0-9]) {1,2}[0-9()]{2,5}",""]# remove trailing number  ex. 90 g/mol 1999 ->  90 g/mol]importunit_parsemore_last_minute_sub=[["MOL","mol"]]# [bad text/regex, new text]unit_parse.config.last_minute_sub+=more_last_minute_sub# Here we are adding to the existing listresult=unit_parse.parser("100 MOL")# pint is case-sensitive, so this will result in an invalid unitprint(result)# Quantity("100 mole")NotesPint UnitRegistryPint's requires a Unit Registry to be defined. However, Unit Registries are not interoperable and will throw
errors if a unit from one registry is used in another. Unit_Parse will go looking to see if one has been created,
and if it hasn't we will make one!So if your project uses Pint already, make sure you import Pint and define theUnitRegistrybefore
importingunit_parse. You must also defineUnitandQuantityto make the registry discoverable.importpintu=pint.UnitRegistry()U=Unit=u.UnitQ=Quantity=u.Quantityfromunit_parseimportparser# your code from here…