Spec 01 · Semantic Compression Layer · Patent Pending
CLM logo

Language, reduced to its semantic core.

CLM extracts structured meaning from system prompts, conversation threads, and data catalogs — in sub-millisecond time, without a model in the loop. Run it standalone as a compression layer, or place it in front of any LLM to cut what you send it by up to 85%.

Start for free →Read the spec10
91.5%
encoder accuracy
4.2M+
daily compressions
<1ms
compression latency
FIG. 1 — support thread
NATURAL LANGUAGE
Hi support team, I noticed my account was charged twice this month — one on the 2nd and another on the 3rd. Can you please look into this? My account email is melissa.jordan@example.com. Thanks, Melissa --- Hi Melissa, Thanks for reaching out. I can confirm the duplicate charge — it was caused by a payment retry that fired after the first transaction already succeeded. I've initiated a full refund on the second charge. You'll see it within 3–5 business days. Your reference number is RFD-908712. Best, Raj – Support Team
ORIGINAL131 tokens
COMPRESSED81 tokens
REDUCTION
Measured in production, not benchmarked in a lab
27%
avg. token reduction, blended across the full daily traffic mix
1B+
tokens encoded per day in production
<1ms
p50 encode latency per document

Best-case reduction on a single well-structured transcript reaches 85%+ — that's the ceiling. 27% is the number that survives contact with everything else: short threads, mixed languages, malformed input, edge cases nobody optimizes for. It's the figure we'd defend in a review.

Spec 03 · Applications

One compression pass, seven downstream reads.

Each entry reads off the same compressed output. Pair any of them with an LLM if you want one — none of them need it to run.

00QA FIELDS

Every thread reduces to a QA-ready field set

Resolution state, sentiment, agent actions, and commitments come out as structured fields the instant a thread closes — plug your own rubric or model on top.

state:RESOLVED
sentiment:GRATEFUL
agentActions:REFUND_INITIATED
01COMMITMENT TRACKING

Catch commitments before they slip

CLM surfaces explicit commitments ("demo by Friday", "proposal in 48h", "refund in 3–5 days") as structured fields you can track and alert on — support, sales, or ops.

commitments:DEMO_BY_FRIDAY
owner:REP
state:PENDING
02SIGNAL EXTRACTION

Intent, stage, and language — extracted in microseconds

Structured fields for intent, stage, language, and sentiment are ready before any downstream system touches the thread — combine them into your own routing logic.

intent:EVALUATE_PRICING
dealStage:DISCOVERY
lang:EN
03AGGREGATE ANALYTICS

Aggregate trends across millions of threads

Because every output is a structured dict, grouping by intent, domain, or sentiment is a single SQL query — no embeddings, no clustering.

intent:SCHEDULE_FOLLOWUP
sentiment:POSITIVE
channel:CALL
04SCHEMA MAPPING

Auto-populate fields from every interaction

Structured output maps directly to your own schemas — domain, service, trigger, context entities — no brittle regex or form-fills required.

domain:INFRA
service:PAYMENTS_API
trigger:LATENCY_SPIKE
05COMPLIANCE REVIEW

Structured fields for compliance review

CLM extracts PII artefacts, commitments, and conversation state as structured fields — giving compliance teams a clean, reviewable feed instead of raw transcripts, whether the source is a support call, sales negotiation, or advisory session.

context:CONSENT_CONFIRMED · REF_ID
commitments:FOLLOWUP_CALL
state:RESOLVED
06KNOWLEDGE MINING

Surface recurring issues across any process

Group by trigger + domain to find what comes up most — then generate runbooks, FAQs, or self-serve docs from real interaction data.

trigger:LATENCY_SPIKE
domain:INFRA
secondaryIntent:ROLLBACK_REQUESTED
Spec 04 · Architecture

One framework, three encoders.

CLM compresses the three content types that drive most LLM cost: system prompts, conversation threads, and structured catalogs.

System Prompt Encoder
Converts verbose natural language instructions into compact semantic vocabulary — REQ, TARGET, EXTRACT, CTX, and OUT — without losing enforcement behavior.
SAMPLE OUTPUT
[PROMPT_MODE:CONFIGURATION][ROLE:CUSTOMER_SUPPORT_AGENT][RULES:BASIC,CUSTOM][PRIORITY:CUSTOM_OVER_BASIC][OUT_JSON:{greeting:STR,response:STR,next_steps:[STR]}] <basic_rules> - Detect input language automatically - Apply appropriate grammar and style - Improve clarity and readability - Output only the enhanced text </basic_rules> <custom_rules> - Always greet the customer by name: {{customer_name}} - Reference their account type: {{account_type}} - Use {{tone}} tone throughout the conversation </custom_rules>
AVG TOKEN REDUCTION~65%
Thread Encoder
Compresses full conversation transcripts into structured interaction summaries — sentiment arcs, agent actions, and commitments preserved.
SAMPLE OUTPUT
[[INTERACTION:SUPPORT:CHANNEL=VOICE] [LANG=EN] [DOMAIN:TECHNICAL] [CUSTOMER_INTENT:REPORT_SERVICE_OUTAGE] [INTERACTION_TRIGGER:CONNECTIVITY_DROPS] [CONTEXT:NAME_PROVIDED] [AGENT_ACTIONS:DIAGNOSTIC_PERFORMED→TROUBLESHOOT→FOLLOWUP_SCHEDULED] [RESOLUTION:PENDING] [STATE:PENDING_ENGINEERING] [COMMITMENT:FOLLOWUP_2H] [COMMITMENT:FOLLOWUP_TOMORROW] [SENTIMENT:NEUTRAL→GRATEFUL
AVG TOKEN REDUCTION~90%
Structured Data Encoder
Collapses structured JSON catalogs into header-plus-row encoding. Preserves IDs, priorities, prerequisites, and action chains.
SAMPLE OUTPUT
{article_id,title,content,category,tags,views}[KB-001,How to Reset Password,To reset your password; go to the login page and click...,Account,password+security+account,1523]
AVG TOKEN REDUCTION~92%
Spec 05 · Verification

Every compression is checked, not assumed.

The CLM quality gate runs three weighted entropy analyses on every compressed output to certify semantic losslessness before it reaches your LLM.

25%
Kolmogorov complexity
Structural complexity via compression ratio — checks whether we simplified too far.
50%
Conditional entropy
Slot coverage validation — confirms H(structured | compressed) collapses to zero.
25%
Perplexity delta
Model comprehension test — compares natural-language and compressed-language responses.
91.5%COMPOSITE CONFIDENCE · VERDICT: ACCEPTABLE
quality_gate.py
from clm_core import CompressionQualityGate

gate = CompressionQualityGate()
report = gate.analyze(
original=encoded.original,
compressed=encoded.compressed,
structured=encoded.to_dict(),
)

# → Verdict: ACCEPTABLE
# → Confidence: 91.5%
Spec 06 · Quickstart

From natural language to compressed tokens, one call.

import clm

client = clm.Client(api_key="YOUR_API_KEY")

result = client.encode(
    text="You are a customer service AI. Analyze "
         "the transcript and extract the main issue, "
         "sentiment, and agent actions. Return JSON.",
    data_type="SYSTEM_PROMPT"
)

print(result.compressed)
# [REQ:ANALYZE>EXTRACT] [TARGET:TRANSCRIPT]
# [EXTRACT:ISSUE+SENTIMENT+AGENT_ACTIONS]
# [OUT:JSON]

print(f"Reduced {result.n_tokens} -> {result.c_tokens} tokens")
# Reduced 148 -> 19 tokens  (87% reduction)
Live playground
View on GitHub →Read the docs →