Yu-and-Ai/yutabase-reposearch-minieval huggingface dataset:
Query (conceptual-boundary, easy): Is YUTABASE a database engine, and which responsibilities remain PostgreSQL's?
Expected facts:
YUTABASE is a candidate semantic profile, not a database engine.
PostgreSQL remains responsible for storage, transactions, access control, backup, and replication.
Relevance judgments:
grade: 3
rationale: Direct project-level definition and substrate boundary.
grade: 3
rationale: Normative candidate statement of the same boundary.
grade: 2
rationale: Research note restates what Core serves and deliberately leaves outside.
grade: 1
rationale: Lists operational responsibilities the candidate does not provide.
Query (conceptual-mapping, easy): BOOK、DECK、CARD、THREAD 同 LEXICON 喺 PostgreSQL 入面各自係咩？
Expected facts:
BOOK is a logical namespace, DECK a registered record collection, CARD a row, THREAD an active directed relation, and LEXICON the governed relation vocabulary.
Relevance judgments:
grade: 3
rationale: Normative table maps all five primitives to PostgreSQL representations.
grade: 2
rationale: Explanatory table corroborates all five mappings.
Query (claim-boundary, medium): `at/by/how/src` 係 row-level claim 定係 identity／truth proof?
Expected facts:
The header is a self-reported claim about a whole row.
It is not proof of truth, identity, authorization, consent, or field-level provenance.
Relevance judgments:
grade: 3
rationale: Normatively defines the header and its non-proof boundary.
grade: 2
rationale: Plain-language corroboration and version-pinning context.
grade: 2
rationale: Applies the row-level claim boundary to projected source statements.
grade: 2
rationale: Adapter guidance warns that the header does not substitute for identity or policy evidence.
Query (validation-rule, medium): `how=cached` 或 `computed` 時，`src` 要滿足咩條件？
Expected facts:
Cached and computed claims require at least one source locator.
Every source entry must be non-null and non-blank, with the candidate's portable array shape.
Relevance judgments:
grade: 3
rationale: Normative source-locator and canonical-array requirements.
grade: 3
rationale: SDK implementation rejects missing and blank source entries.
grade: 2
rationale: SDK guide states the same programmatic contract.
grade: 2
rationale: Projection profile specifies stable source material for copied and derived rows.
Query (conformance-environment, easy): Which PostgreSQL, Node, and Bun versions are covered by candidate conformance?
Expected facts:
The Postgres-profile claim covers PostgreSQL 16 and 17.
The SDK declares Node.js 20 or newer, while candidate development and CI are pinned to Bun 1.3.5.
Relevance judgments:
grade: 3
rationale: Exact supported environment and qualification.
grade: 2
rationale: Normative summary confirms the PostgreSQL major boundary.
grade: 1
rationale: Project overview separately warns against PostgreSQL support claims outside 16 and 17.
Query (integration-boundary, medium): Annex 現有 PostgreSQL table 會唔會搬 rows 或自動 grant application-table access？
Expected facts:
Annexing registers a compatible existing table without moving its rows.
It grants no application-table privileges; ordinary PostgreSQL access remains separately required.
Relevance judgments:
grade: 3
rationale: Complete integration contract, authority requirements, and non-grant boundary.
grade: 2
rationale: SDK-facing summary corroborates row and privilege behavior.
Query (code-behavior, hard): How does `summarizeFreshness` avoid double-counting a traversal's final edge, and what happens when no claim values are found?
Expected facts:
Traversal path claims are counted instead of the repeated top-level final edge.
The function returns undefined when no values with a claim kind are found.
Relevance judgments:
grade: 3
rationale: Direct implementation and comments answer both branches.
grade: 2
rationale: Shows how a valid traversal path is recognized.
grade: 2
rationale: SDK guide explains edge counting and malformed timestamp handling.
Query (future-security-boundary, medium): Post-quantum 入面 ML-KEM 同 ML-DSA／SLH-DSA 分工係咩？Private key 可唔可以放入 YUTABASE？
Expected facts:
ML-KEM is for key establishment, while ML-DSA and SLH-DSA are signature schemes.
The proposed evidence annex must keep private keys, seeds, bearer tokens, recovery phrases, and database credentials outside its decks.
Relevance judgments:
grade: 3
rationale: Direct non-normative decision record for algorithm roles and custody.
grade: 2
rationale: High-level decision retains post-quantum continuity while keeping private keys outside.
Query (conceptual-boundary, medium): Should KARMA become a transferable reputation or moral score? What is the narrower database shape?
Expected facts:
No canonical KARMA primitive is minted.
The narrow shape is action to observed consequence to acknowledgement to correction or repair, represented as evidence-linked claims rather than a score.
Relevance judgments:
grade: 3
rationale: Directly defines the proposed boundary and rejected scoring model.
grade: 2
rationale: The explicit not-yet list rejects KARMA and related worth or goodness scores.