DISTILBERT RUNNING ON DEEPSPARSE GOES BRHMMMMMMMM. πππ
This model is π
ββββββββ βββββββ ββββββ βββββββ ββββββββ ββββββββ
ββββββββ ββββββββ ββββββββ ββββββββ ββββββββ ββββββββ
ββββββββ ββββββββ ββββββββ ββββββββ ββββββββ ββββββ
ββββββββ βββββββ ββββββββ ββββββββ βββββββββ βββββ
ββββββββ βββ βββ βββ βββ ββ βββββββββ ββββββββ
ββββββββ βββ βββ βββ βββ ββ βββββββββ ββββββββ
LOOKS LIKE THIS π
Inference endpoints, outside of outliers (4ms) is avg. latency on 2 vCPUs:
Handler for access to inference endpoints
class EndpointHandler:
def __init__(self, path=""):
self.pipeline = Pipeline.create(task="text-classification", model_path=path)
def __call__(self, data: Dict[str, Any]) -> Dict[str, str]:
"""
Args:
data (:obj:): prediction input text
"""
inputs = data.pop("inputs", data)
start = perf_counter()
prediction = self.pipeline(inputs)
end = perf_counter()
latency = end - start
return {
"labels": prediction.labels,
"scores": prediction.scores,
"latency (secs.)": latency
}
Μ·ΜΝ Μ΅ΝΜ³RΜΆΝΜiΜΈΝΜcΜ΄ΜΜ»kΜΈΜΝyΜ·ΝΜ³ ΜΈΜΜͺ Μ·ΝΝ
- Downloads last month
- 1