wolfrage89/company_segment_ner

Roberta based NER

This model will take in a new article label 3 entities [ORGS, SEGNUM, NUM]. This model is train on reuters news articles

Try out on huggingface Spaces

https://huggingface.co/spaces/wolfrage89/company_segments_ner

colab sample notebook

https://colab.research.google.com/drive/165utMQzYVAX7-aQjWjpmPHwHpdKTaHBa?usp=sharing

How to use

from transformers import pipeline

# Minimum code
sentence = """Exxon Mobil Corporation is engaged in energy business. The Company is engaged in the exploration, production, trade, transportation and sale of crude oil and natural gas, and the manufacture, transportation and sale of crude oil, natural gas, petroleum products, petrochemicals and a range of specialty products. The Company's segments include Upstream, Downstream, Chemical, and Corporate and Financing. The Upstream segment operates to explore for and produce crude oil and natural gas. The Downstream manufactures, trades and sells petroleum products. The refining and supply operations consists of a global network of manufacturing plants, transportation systems, and distribution centers that provide a range of fuels, lubricants and other products and feedstocks to its customers around the world. The Chemical segment manufactures and sells petrochemicals. The Chemical business supplies olefins, polyolefins, aromatics, and a variety of other petrochemicals."""


model = pipeline('ner', "wolfrage89/company_segment_ner")
model_output = model(sentence)

print(model_ouput)
# [{'entity': 'B-ORG', 'score': 0.99996805, 'index': 1, 'word': 'Ex', 'start': 0, 'end': 2}, {'entity': 'I-ORG', 'score': 0.99971646, 'index': 2, 'word': 'xon', 'start': 2, 'end': 5}, ....]


# Sample helper function if you want to use 
def ner_prediction(model, sentence):
    entity_map = {
        "B-ORG":"ORG",
        "B-SEG":"SEG",
        "B-SEGNUM":"SEGNUM"
    }
    results = []
    model_output = model(sentence)

    accumulate = ""
    current_class = None
    start = 0
    end = 0
    for item in model_output:
        if item['entity'].startswith("B"):
            if len(accumulate) >0:
                results.append((current_class, accumulate, start, end))
            accumulate = item['word'].lstrip("Ġ")
            current_class = entity_map[item['entity']]
            start=item['start']
            end = item['end']
            
        else:
            if item['word'].startswith("Ġ"):
                accumulate+=" "+item['word'].lstrip("Ġ")
                
            else:
                accumulate+=item['word']
            end = item['end']

    # clear last cache
    if len(accumulate)>0:
         results.append((current_class, accumulate, start, end))

    return results