jacob-valdez/tensorcode-investigator-cognitive-experimental-001

TensorCode Investigator: experimental cognitive checkpoint

Owns hypothesis generation, source verification, ranking and sentence retrieval. This is the exact Investigator component extracted from the evaluated Chatbot; it excludes the outer realizer and has no separate end-to-end answer benchmark.

Install from the source commit below with python -m pip install -e '.[tools]'.

from tensorcode.tools.investigator import Investigator
model = Investigator.from_pretrained("jacob-valdez/tensorcode-investigator-cognitive-experimental-001")  # Pin a Hub commit for reproducibility.
result = model.investigate({"question": "What does this source say?", "evidence": [{"id": "note", "source_id": "user:note", "text": "The service recovered after reconnecting the database."}]})
print(result)

Status and measured limits

Experimental research artifact, not a dependable reasoning assistant. The complete Chatbot evaluation on 32 fixed HotpotQA questions with oracle supporting passages returned 30 abstentions, one source-reviewed correct answer and one circular non-answer. Both returned answers were manually reviewed. Literal short-answer exact match was zero because the correct answer was expressed as a full sentence. The circular response passed NLI despite not supplying the requested age.

All eight evidence-omission and eight replacement controls abstained. Given the high original-evidence abstention rate, this alone does not establish counterfactual reasoning. Generation and verification inherit fallible foundation capabilities; source support does not establish truth, relevance or answer completeness.

An owned MiniLM encoder retrieved a supporting passage at rank 1 for all 32 queries on this small oracle-passage corpus (lexical baseline 31/32); both reached 32/32 at rank 5. This is inherited sentence-retrieval ability, not a retrieval fine-tune or open-domain benchmark. Actual developer-document evidence survived episodes and session save/load and was retrieved, but its question still received an abstention.

Components and training

• Generator: google/flan-t5-small at 0fc9ddf78a1e988dac52e2dac162b0ede4fd74ab, adapted for 3 epochs on 1,024 human QA2D declarations. Only original questions and SQuAD paragraphs enter generator inputs; targets are excluded. Article-disjoint test declaration match improves from 1/128 to 30/128, but factual errors remain.
• Ranking: the prior TensorCode Electra-small HotpotQA document ranker. Its labels concern document relevance; proposal-quality ranking is not separately learned.
• Verifier: cross-encoder/nli-deberta-v3-small at fa2804872c3b4bd748f38c0185cc85775361e735, 1,024 SNLI fine-tuning pairs, 256 separate temperature-fitting pairs, 256 test pairs. Temperature 1.9768; fine-tuning accuracy 92.58% to 92.19%, calibrated NLL 0.2420, ECE 0.0286. Foundation already used SNLI/MultiNLI. Most human QA2D declarations fail the applied support policy even without truncation: calibration does not transfer automatically to every domain.
• Retrieval: sentence-transformers/all-MiniLM-L6-v2 at 1110a243fdf4706b3f48f1d95db1a4f5529b4d41, unchanged pretrained weights, masked mean pooling, L2 normalization, 256-token limit, no random projection.

The Chatbot additionally owns a google/flan-t5-base realizer at 7bcac572ce56db69c1ea7c8af255c5d7c9672fc2, adapted for 3 epochs on 256 selected human statements. The selected statement intentionally occurs in the input: this is faithful rendering, not target-blind QA. Normalized statement match on 64 article-disjoint development examples reaches 100% (verbatim 96.875%).

The support policy is authored: support >=0.7, contradiction <=0.2, unknown <=0.3, with truncation vetoes. Memory capacity is 256, top-k 5, cognitive record capacity 1024 in the Chatbot. Model weights are separate from application session state. Generated proposals and assistant responses are never automatically observations. Symbolic graph operations remain unimplemented. No consistent benefit from the recurrent workspace is demonstrated by these results.

Data, reproducibility and license notices

Full source texts, outputs, scores, split IDs and controls are in evaluation.json. The model/configuration/component hashes were frozen before final data selection; final-freeze.json records that boundary. Final data is HotpotQA distractor validation[272:304], revision 1908d6afbbead072334abe2965f91bd2709910ab. Prior validation[256:272] was used for diagnosis and component changes, not treated as an untouched final test. Foundation pretraining exposure is unknown.

QA2D source: domenicrosati/QA2D revision d38d3f42978e72c8c3ccc5dca0d3a2ac745f1fcf, human turker_answer, joined to rajpurkar/SQuAD-explorer revision eee5fdbf62f8613a7812b03419e6b29617b74fd1. SQuAD material carries CC-BY-SA-4.0; retain source attribution and applicable data license terms. Foundation model licenses and data terms remain applicable. See component training reports and original dataset/model repositories for details. TensorCode implementation is MIT licensed. All substantial training and real-model inference for this release ran on the connected GB10, not the development host.

Code: https://github.com/TensaCo/tensacode-py at commit b26785d (TensorCode 0.3.0a1 source checkout). Reports preserve historical executed-script hashes; reporting-only script corrections followed training. This is a data-only native TensorCode artifact with complete owned tensor weights and tokenizer configuration; no custom Hub Python code is executed.