semvec/pypi-packages huggingface dataset:
jfinqa-helm 0.1.0
HELM plugin for JFinQA: Japanese Financial Numerical Reasoning QA Benchmark
License: Apache-2.0
Requires Python: >=3.9
Keywords: benchmark, finance, helm, japanese, nlp

# jfinqa-helm

HELM plugin for [JFinQA](https://github.com/ajtgjmdjp/jfinqa): Japanese Financial Numerical Reasoning QA Benchmark.

## Installation

```bash
pip install crfm-helm jfinqa-helm
```

## Usage

```bash
helm-run --run-entries jfinqa:model=openai/gpt-4o
```

## About JFinQA

JFinQA contains 1,000 questions across three subtasks:

- **numerical_reasoning** (550 questions): Calculate financial ratios, growth rates, etc.
- **consistency_checking** (200 questions): Verify whether a statement is consistent with financial data
- **temporal_reasoning** (250 questions): Reason about changes over multiple fiscal years

Questions are drawn from 68 companies' EDINET filings covering J-GAAP, IFRS, and US-GAAP.

**Dataset**: [ajtgjmdjp/jfinqa](https://huggingface.co/datasets/ajtgjmdjp/jfinqa)

## License

Apache-2.0