Investigating Knowledge Graphs for Context-Aware Search in Financial Transactional AI Agents
| dc.contributor.author | Baisenkul N. | |
| dc.date.accessioned | 2026-10-27T05:58:41Z | |
| dc.date.available | 2026-10-27T05:58:41Z | |
| dc.date.issued | 2026 | |
| dc.description.abstract | This thesis presents an empirical evaluation of a hybrid knowledge-graph and largelanguage-model (KG–LLM) architecture for financial transactional compliance reasoning. The architecture was proposed in a companion conference paper [1] that established the design and demonstrated it on a small worked example; the contribution of this thesis is the quantitative, ablation-based, robustness-aware empirical study that completes the proposal. A deterministic synthetic dataset of 500 transactions across eight compliance-rule scenarios plus a clean negative class is generated under two complementary disciplines: a crossrule labeling pass that mirrors how real anti-money-laundering pipelines assign overall compliance, and an explicit per-channel audit of base-context fields for label-correlated content (which uncovered, and fixed, a substantial description-field scenario-leakage problem in the dataset’s earlier versions; the empirical impact of the leakage and the corrected results are reported throughout). A four-arm ablation matrix isolates the contribution of structured KG relations by varying only the grounding mode supplied to the language model. A parametric fifth arm sweeps per-transaction KG coverage from 100 % down to 0 % to characterise the architecture’s degradation under realistic incomplete-graph conditions. On the corrected, description-neutral dataset, the KG-augmented arm achieves accuracy 0.891 with perfect precision (1.000, zero wrongly-cleared violations across 1500 predictions), recall 0.691, and F1 0.817 across three independent repeats. Both no-KG baselines—a vanilla LLM and a rule-text-grounded LLM—collapse to majority-class-default behaviour at F1 of approximately 0.01, so the architecture roughly fifty-fold lifts F1 relative to the baselines while never wrongly clearing a violation. The KG-augmented arm’s accuracy advantage degrades smoothly with KG coverage and disappears at coverage below approximately 20 % (the dropout sweep crosses the no-KG baseline at p ∗ ≈ 0.81). The methodological contributions of the work include the rule-name neutrality discipline—a previously-undocumented prompt-design pitfall whereby loaded rule identifiers override structured per-rule relations in language-model reasoning, identified during the project’s first benchmark and described in detail as finding F-001 in the project record—and the description-channel audit protocol (F-007) that the thesis argues should be a default hygiene step in any synthetic-data pipeline feeding a language model. | |
| dc.identifier.citation | Baisenkul N / Investigating Knowledge Graphs for Context-Aware Search in Financial Transactional AI Agents / SDU University / Department of Computer Science | |
| dc.identifier.uri | https://repository.sdu.edu.kz/handle/123456789/2693 | |
| dc.language.iso | en | |
| dc.publisher | SDU University | |
| dc.subject | knowledge graphs | |
| dc.subject | large language models | |
| dc.subject | retrieval-augmented generation | |
| dc.subject | financial AI agents | |
| dc.title | Investigating Knowledge Graphs for Context-Aware Search in Financial Transactional AI Agents | |
| dc.type | Thesis |