Lawbot Query Metadata

What we capture about each user query today · Prod Lawbot · 19 Sep – 19 Oct 2026 · 7,707 queries

Short answer

Yes

Query difficulty: every query is labelled EASY, MEDIUM or COMPLEX.

Yes

With doc / without doc: we know whether the user uploaded a PDF to the conversation.

Yes

Query intent: new question vs follow-up, case-law focus, whether a document search was needed, whether the answer needs a table.

Partly

Small / large query: no length field, because query text is stored encrypted. Difficulty works as the size label instead: a COMPLEX query uses about 2.5× the tokens and cost of an EASY one (see below).

What we track per query

Metadata	Values	Source (TaxmannDW)
Difficulty	EASY · MEDIUM · COMPLEX	queries_from_lawbot.query_classification → Difficulty_level
Query type	New query · Follow-up · Tariff research · Tariff search · ITA comparative · Chit-chat	queries_from_lawbot.query_classification → flag
Uploaded a document (PDF)	Yes · No	threads_from_lawbot.is_pdf_thread
Document search needed	Retrieve · No retrieve	queries_from_lawbot.query_classification → retrieval_needed
Case-law focused	Yes · No	queries_from_lawbot.query_classification → focus_caselaws
Answer needs a table	Yes · No	queries_from_lawbot.query_classification → table_flag
Topic	Short auto-generated label	queries_from_lawbot.query_classification → chat_topic
Law area (user choice)	Auto · Income Tax · GST	queries_from_lawbot.applied_filters → law
Answer length (user choice)	Long · Short	queries_from_lawbot.applied_filters → format_type
Sources searched (user choice)	Act / Rules, Case laws, Books, Articles, …	queries_from_lawbot.applied_filters → datasets
Tokens and cost	Per-query input / output tokens by model	queries_from_lawbot.tokens
Outcome	Success · Failed · Edited · Regenerated	queries_from_lawbot.is_success, is_edited, is_regenerated

Last 30 days

Difficulty of 6,923 labelled

Medium

49% · 3,415

Complex

36% · 2,518

Easy

14% · 990

Query type of 6,927 labelled

New query

64% · 4,446

Follow-up

29% · 2,032

Tariff research / search

3% · 228

Chit-chat

2% · 151

ITA comparative

1% · 70

Document uploaded of 7,707

Without doc

91% · 7,004

With PDF

9% · 703

User choices of 7,707

Law: Auto

47% · 3,603

Law: Income Tax

42% · 3,256

Law: GST

11% · 848

Long answer

92% · 7,099

Short answer

8% · 608

Size by category · average per query

Category	Queries	LLM calls	Input tokens	Output tokens	LLM cost
Easy	985	21	490K	19K	$0.26
Medium	3,389	39	1.09M	44K	$0.56
Complex	2,502	42	1.21M	53K	$0.65
Without doc	6,258	39	1.10M	46K	$0.57
With PDF	623	22	566K	23K	$0.33

Labelled queries with token data. Tokens and cost are summed over every LLM call a query makes, from queries_from_lawbot.tokens → llm_stats[].input_tokken, output_tokken, stage_cost (USD). Most input tokens are retrieved legal content, not the user's typed text.

Gaps

No length metric. Query text is encrypted (7,702 of 7,707 rows), so “small vs large” has to come from difficulty or token counts.
10% of queries are unlabelled. The 779 failed queries have no classification.
Enterprise (API) queries are mostly unlabelled. Only 138 of 1,091 have a classification.
Labels are produced by the LLM. They work for reporting but aren't validated, and a few rows have misspelled field names.

Short answer

What we track per query

Last 30 days

Difficulty of 6,923 labelled

Query type of 6,927 labelled

Document uploaded of 7,707

Other flags share answering “yes”

User choices of 7,707

Size by category · average per query

Gaps