🚀 Models ⚡️ Inference API
🔥 Sign in
Back to all models
Model: sagorsarker/codeswitch-hineng-lid-lince
pytorch bert token-classification hi en dataset:lince 🏷 license:mit codeswitching hindi-english language-identification
token-classification mask_token: [MASK]
Query this model
🔥 This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚡️ Upgrade your account to access the Inference API 

							$
							curl -X POST \

								-H "Authorization: Bearer YOUR_ORG_OR_USER_API_TOKEN" \

							-H "Content-Type: application/json" \

								-d '"json encoded string"' \

							https://api-inference.huggingface.co/models/sagorsarker/codeswitch-hineng-lid-lince
Share Copied link to clipboard

Monthly model downloads

sagorsarker/codeswitch-hineng-lid-lince sagorsarker/codeswitch-hineng-lid-lince
67 downloads
last 30 days

pytorch

tf

Contributed by

sagorsarker Sagor Sarker
9 models

How to use this model directly from the 🤗/transformers library: 

			
Copy to clipboard

			from transformers import AutoTokenizer, AutoModelForTokenClassification

			tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince")
			
			model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince")
List all files in model · See raw config file
Update on GitHub

codeswitch-hineng-lid-lince

This is a pretrained model for language identification of hindi-english code-mixed data used from LinCE

This model is trained for this below repository.

https://github.com/sagorbrur/codeswitch

To install codeswitch:

pip install codeswitch

Identify Language

  • Method-1

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince")

model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince")
lid_model = pipeline('ner', model=model, tokenizer=tokenizer)

lid_model("put any hindi english code-mixed sentence")
  • Method-2
from codeswitch.codeswitch import LanguageIdentification
lid = LanguageIdentification('hin-eng') 
text = "" # your code-mixed sentence 
result = lid.identify(text)
print(result)
List of model files
File name Last modified File size
config.json Sat, 22 Aug 2020 06:03:11 GMT 991.0B
eval_results.txt Sat, 22 Aug 2020 06:03:03 GMT 183.0B
pytorch_model.bin Sat, 22 Aug 2020 06:03:04 GMT 678.6MB
special_tokens_map.json Sat, 22 Aug 2020 06:03:02 GMT 112.0B
test_predictions.txt Sat, 22 Aug 2020 06:03:03 GMT 124.2KB
test_results.txt Sat, 22 Aug 2020 06:03:03 GMT 171.0B
tokenizer_config.json Sat, 22 Aug 2020 06:03:04 GMT 49.0B
training_args.bin Sat, 22 Aug 2020 06:03:11 GMT 1.5KB
vocab.txt Sat, 22 Aug 2020 06:03:12 GMT 972.2KB
Done