Aggregator Integration Service¶

aggregator-integration-service connects the SMS Gateway to Infozillion/MNPSP. It reads SMS chunks created by sms-router-service, selects the carrier credentials and CLI, sends the SMS, checks delivery reports when needed, updates recipient status, and starts refunds for eligible direct messages.

The service runs in the background. It has no business REST API for external or internal clients. It receives work through Kafka and makes HTTP calls to Infozillion and the internal refund endpoint in client-service. Spring Boot Actuator is for operations only and must remain private.

Read this page with the aggregator process flows, configuration reference, SMS router architecture, and SMS router process flows.

System context¶

flowchart LR
    Client[API consumer] --> Gateway[gateway-service]
    Gateway --> Router[sms-router-service]
    Router -->|deduct / campaign refund| Ledger[client-service]
    Router -->|commit chunks, recipients, outbox| RouterDb[(sms_router schema)]
    RouterDb --> RouterPublisher[Router OutboxPublisher]
    RouterPublisher -->|sms.outbound.mnoUserId| Kafka[(Kafka)]
    Kafka -->|consume executable chunk| Aggregator[aggregator-integration-service]
    Aggregator -->|POST send-sms / check-delivery-report| Infozillion[Infozillion / MNPSP]
    Aggregator -->|update chunks and recipients| RouterDb
    Aggregator -->|audit and refund log| AggregatorDb[(sms_aggregator schema)]
    Aggregator -->|direct-send refund over HTTP| Ledger
    Aggregator -->|insert MESSAGE_FINALIZED| RouterDb
    RouterPublisher -->|sms.reporting| Kafka
    Kafka -->|reporting consumer| Ledger

Hold "Ctrl" to enable pan & zoom

Each service has a clear role:

The router validates and normalizes requests, calculates and charges the cost, creates chunks, manages campaigns, and publishes outbox events to Kafka.
The aggregator calls Infozillion and interprets carrier send and DLR responses.
client-service owns the monetary ledger and reporting facts.
The aggregator can update a limited set of router tables because dispatch and DLR data is shared with message status and campaign processes.

Responsibilities and exclusions¶

The aggregator owns	The aggregator does not own
Read an SMS chunk from Kafka and process it	Public authentication or identifying the client
Check primary or secondary sender routing before dispatch	SMS validation, segmentation, pricing, or the first charge
Submit MNO and IPTSP requests to Infozillion	Campaign scheduling, pause/resume/cancel, or BTRC-window enforcement
Interpret send response codes and DLR results	Publishing the original outbound event
Update chunk/recipient execution state	Campaign status aggregation and campaign refunds
Write carrier-call audit data	Store or expose an application API
Register and retry eligible direct-message refunds	Publish reporting events directly to Kafka
Insert terminal reporting events into the router outbox	Manage routing or billing configuration

Runtime components¶

Component	What it does
`SmsChunkConsumer`	Reads `sms.outbound.<mnoUserId>`, parses and validates `SmsChunkEvent`, and starts dispatch.
`KafkaConfig`	Retries failed records with increasing delays and sends records that still fail to the related `.dlt` topic.
`DltConsumer`	Reads outbound DLT records in a separate consumer group. It currently only logs them.
`MnpspDispatcher`	Claims a chunk, selects the final route, calls Infozillion outside a database transaction, and saves the result in one transaction.
`EffectiveSenderRouteResolver`	Applies exact or wildcard CLI rules and selects the configured primary or generated secondary route.
`MnpspClient`	Builds MNO or IPTSP URLs, makes blocking `WebClient` POST calls, applies timeouts, and uses separate circuit breakers for sending and DLR checks.
`DlrPoller`	Claims ready `SUBMITTED` recipients, groups them by `serverTxnId`, checks delivery, and schedules the next check.
`ReportFinalizedPublisher`	Adds one `MESSAGE_FINALIZED` event for each final recipient to `sms_router.outbox_event`.
`RefundService`	Creates idempotent direct-message refund records for a chunk or recipient and calls `client-service`.
`RefundRetryScheduler`	Retries `PENDING` refunds and stops after ten attempts.
`TpsLimiter`	Defines a local Caffeine counter for each credential, but dispatch does not currently use it.

Inbound Kafka contract¶

Topic and consumer semantics¶

The router publishes each chunk to sms.outbound.<mnoUserId>. The main consumer uses the full-match regular expression sms\\.outbound\\.[^.]+. The last part cannot contain a dot, so it excludes DLT topics such as sms.outbound.42.dlt. This prevents the main consumer from reading a failed record from the DLT and sending the SMS again.

The Kafka key is the chunk ID, so repeated events for the same chunk keep their order. Kafka cannot make the external HTTP call exactly once. The chunk claim stored in the database prevents duplicate sends.

`SmsChunkEvent`¶

The router's OutboxChunkPayload and the aggregator's SmsChunkEvent are separate code copies of the same JSON format. The consumer ignores unknown fields so a producer can add fields without breaking it.

Field	Required by consumer validation	Purpose
`chunkId`	Yes	Router chunk ID and the identity used to prevent duplicate dispatch.
`smsRequestId`	No	Links the event to its parent request.
`messageId`	No	Public message ID.
`senderConfigId`	No	Saved public sender configuration. A null value uses legacy routing.
`mnoId`	Yes	MNO group selected by the router; also used by legacy routing.
`operator`	No	`GP`, `RB`, `BL`, or `TT` snapshot.
`mnoUserId`	Yes	Credential selected by the router and the outbound topic suffix.
`senderCli`	No	Requested CLI and the key used to find its routing rule.
`smsType`	No	`OTP`, `TRANSACTIONAL`, or `PROMOTIONAL`; maps to Infozillion transaction type.
`lang`, `encoding`, `segmentCount`	No	Segment details saved by the router. `UCS2` selects message type `3`; other encodings select `1`.
`message`	No	Final message text. Campaign personalization is already complete.
`recipients`	Yes, non-empty	Normalized MSISDNs. All recipients in the chunk use the same route and message settings.
`campaignId`	No	Operator-approved promotional content ID; not the internal campaign identity.
`campaignRef`	No	Internal campaign UUID. Null means a direct SMS, for which the aggregator handles refunds.
`submittedAt`	No	Original router submission time.
`dlrEnabled`	No	`false` finalizes from send response; `true` or null uses normal DLR behavior.
`clientId`, `userId`, `systemUserName`	No	Client and user details used to register direct-message refunds.

Example event:

{
  "chunkId": 72001,
  "smsRequestId": 81001,
  "messageId": "msg_01JZ2Y6E7Y6W4P8M",
  "senderConfigId": 14,
  "mnoId": 2,
  "operator": "GP",
  "mnoUserId": 22,
  "senderCli": "ACME",
  "smsType": "TRANSACTIONAL",
  "lang": "en",
  "encoding": "GSM7",
  "segmentCount": 1,
  "message": "Your payment was received.",
  "recipients": ["8801712345678"],
  "submittedAt": "2026-08-21T10:15:30",
  "dlrEnabled": true,
  "clientId": 42,
  "userId": 7,
  "systemUserName": "acme-api"
}

Invalid JSON or an event missing required data goes directly to the DLT without retry. Other errors use the limited Kafka retry policy.

Effective sender routing¶

For current events, senderConfigId identifies the public route selected and charged by the router. The aggregator reads the route again just before dispatch. This allows an operator to switch a CLI between primary and secondary routes without submitting the message again.

Lock and load the chunk. Reject the event if its senderConfigId does not match the stored value.
Find the sender's exact CLI entry and wildcard * rule. An exact entry with secondary-route settings takes priority. An exact PRIMARY entry that only authorizes the CLI uses the wildcard when one exists; otherwise, it uses the primary route.
Check that the public sender, credential, and MNO configuration are active and consistent.
For activeRoute=PRIMARY, use the public sender's credential and original CLI.
For activeRoute=SECONDARY, require an active generated fallback sender for the same system user and operator. Use its credential and fallbackSenderCli.
Save dispatched_sender_config_id, dispatched_mno_id, dispatched_mno_user_id, dispatched_sender_cli, dispatched_channel_id, and dispatched_channel_name before sending.

The original sender, MNO credential, and channel columns keep the route selected by the router. The dispatched_* columns store the route actually sent to Infozillion. The DLR poller later uses the MNO and credential saved on the chunk.

Legacy events with null senderConfigId do not use the sender graph. They load mnoId and mnoUserId from the event and retain its CLI.

Infozillion integration contract¶

These are calls from the service to the provider. They are not platform APIs.

MNO type	Send URL	DLR URL
`MNO`	`{base}/a2p-sms/api/v1/send-sms`	`{base}/a2p-proxy-api/api/v1/check-delivery-report`
`IPTSP`	`{base}/a2p-sms-iptsp/api/v1/send-sms`	`{base}/a2p-proxy-api-iptsp/api/v1/check-delivery-report`

All calls use POST with JSON. MNO uses mno_user_config.bill_msisdn. IPTSP uses the final sender CLI as billMsisdn because IPTSP credentials do not include a billing MSISDN.

Send request mapping¶

Infozillion field	Source/rule
`username`, `password`	Effective MNO-user credential; password is decrypted by the JPA converter.
`billMsisdn`	MNO credential billing MSISDN, or effective CLI for IPTSP.
`apiKey`	Global `mnpsp.aggregator-api-key`.
`cli`	Primary CLI or configured `fallbackSenderCli`.
`msisdnList`	Chunk recipients.
`transactionType`	`P` for `PROMOTIONAL`; `T` for OTP and transactional.
`messageType`	`3` for `encoding=UCS2`; otherwise `1`. Flash values `2` and `4` are modeled but not selected.
`isLongSMS`	`mno_config.is_long` for the effective route.
`campaignId`	Operator-approved ID supplied by the router.
`message`	Router-finalized message content.

Example, with secrets redacted:

{
  "username": "acme_gp_user",
  "password": "<decrypted-at-runtime>",
  "billMsisdn": "8801700000000",
  "apiKey": "<global-infozillion-key>",
  "cli": "ACME",
  "msisdnList": ["8801712345678"],
  "transactionType": "T",
  "messageType": "1",
  "isLongSMS": false,
  "message": "Your payment was received."
}

Accepted response example:

{
  "serverTxnId": "CP-20260821-00001234",
  "serverResponseCode": 9000,
  "serverResponseMessage": "Success",
  "mnoTxnId": "MNO-9843001",
  "mnoResponseCode": "1000",
  "mnoResponseMessage": "Accepted"
}

DLR request and response¶

Later DLR checks send the send response's serverTxnId as serverReference. Ready recipients with the same transaction ID are grouped into one request.

{
  "username": "acme_gp_user",
  "password": "<decrypted-at-runtime>",
  "billMsisdn": "8801700000000",
  "apiKey": "<global-infozillion-key>",
  "msisdnList": ["8801712345678", "8801712345679"],
  "serverReference": "CP-20260821-00001234"
}

{
  "serverResponseCode": 9000,
  "serverResponseMessage": "Success",
  "serverTxnId": "CP-20260821-00001234",
  "deliveryStatus": [
    "8801712345678-Delivered",
    "8801712345679-UnDelivered"
  ],
  "dndMsisdn": [],
  "invalidMsisdn": [],
  "mnoResponseCode": "1000",
  "mnoResponseMessage": "Processed"
}

MSISDN comparison removes spaces and a leading +. A response may also list recipients in dndMsisdn or invalidMsisdn.

Dispatch consistency model¶

Kafka delivery and an external HTTP request cannot be part of one transaction. When a send result is unclear, the service chooses at-most-once carrier submission:

sequenceDiagram
    participant K as Kafka consumer
    participant DB as PostgreSQL
    participant A as Aggregator
    participant I as Infozillion

    K->>A: SmsChunkEvent
    A->>DB: TX 1, lock and move PENDING/FAILED to DISPATCHING
    DB-->>A: commit durable claim
    A->>I: POST send-sms, no DB transaction
    I-->>A: response or timeout
    A->>DB: TX 2: audit + chunk + recipients + report outbox
    DB-->>A: commit outcome

Hold "Ctrl" to enable pan & zoom

The row lock allows only one consumer to handle a chunk at a time. A repeated event follows these rules:

Stored chunk state	Behavior
`DISPATCHED`	Skip it because the result is already saved.
`DISPATCHING`	Skip it because the previous attempt may have reached the operator and must be checked manually.
Other state, including `FAILED`	Claim as `DISPATCHING` and proceed.
Missing chunk	Skip it.

This prevents duplicate sends after a Kafka rebalance or a database rollback following an accepted provider call. However, a crash after the claim and before saving the result can leave the chunk in DISPATCHING. No current worker fixes this automatically.

Submit outcome policy¶

serverResponseCode=9000 normally means the request was accepted. The next state depends on the MNO type and the client's DLR setting.

Condition	Chunk	Recipient	DLR scheduled	Direct refund
Any send response `9099`	`DISPATCHED`	`DELIVERED`	No	No
`9000`, MNO, DLR on/null	`DISPATCHED`	`SUBMITTED`	First check near +3 minutes	No
`9000`, IPTSP, DLR on/null	`DISPATCHED`	`DELIVERED`	No	No
`9000`, promotional, DLR off	`DISPATCHED`	`DELIVERED`	No	No
`9000`, OTP/transactional, DLR off, MNO code `1000`	`DISPATCHED`	`DELIVERED`	No	No
`9000`, OTP/transactional, DLR off, no MNO code `1000`	`DISPATCHED`	`FAILED`	No	Whole chunk
Non-`9000`, non-`9099`	`FAILED`	`FAILED`	No	Whole chunk
Invalid/missing sender configuration	`FAILED`	`FAILED`	No	Whole chunk

Refunds in the last three rows apply only to direct messages (campaignRef=null). The router handles campaign results and refunds.

The `9099` business decision¶

Infozillion code 9099 means a server failure. The WebClient circuit-breaker fallback also creates a 9099 response for timeouts, connection errors, and an open circuit. Under the current billing rule, every 9099 is treated as delivered and charged, even when the local result is NOT_SENT. Without serverTxnId, the service cannot check the DLR, and it does not issue a refund.

The audit response stores localFallback and fallbackCause (TIMEOUT or NOT_SENT), but the cause does not change the charge. This rule is a billing policy; it does not confirm that the SMS was delivered.

DLR state model¶

Only MNO recipients accepted with DLR enabled enter SUBMITTED. IPTSP, DLR-disabled, rejected, and 9099 results get a final status during submission.

stateDiagram-v2
    [*] --> SUBMITTED: MNO accepted and DLR enabled
    SUBMITTED --> DELIVERED: deliveryStatus=Delivered
    SUBMITTED --> FAILED: deliveryStatus=UnDelivered
    SUBMITTED --> REJECTED: DND or invalid MSISDN
    SUBMITTED --> SUBMITTED: pending, unknown, missing config, or upstream error
    SUBMITTED --> NOT_AVAILABLE: ninth attempt has no terminal result
    DELIVERED --> [*]
    FAILED --> [*]
    REJECTED --> [*]
    NOT_AVAILABLE --> [*]

Hold "Ctrl" to enable pan & zoom

The service checks at about 3 minutes, 10 minutes, 30 minutes, 1 hour, 2 hours, 6 hours, 12 hours, 18 hours, and 23 hours after dispatch. Scheduler timing may add a small delay. If no final result is available after the last check, the status becomes NOT_AVAILABLE, not EXPIRED. There is no refund because delivery is still unknown.

Multiple instances share DLR work with FOR UPDATE SKIP LOCKED, so no scheduler lock is needed across the cluster. The poll method keeps its database transaction open while calling the provider. Larger batches and longer provider timeouts therefore hold locks and database connections for longer.

Refund ownership¶

Failure	Owner	Granularity	Idempotency key
Direct chunk rejected at send or invalid configuration	Aggregator	Entire chunk cost	`chunk:<chunkId>`
Direct recipient DLR is `UnDelivered`, DND, or invalid	Aggregator	Recipient cost	`recipient:<recipientId>`
Direct `NOT_AVAILABLE`	No automatic refund	—	—
Any `9099`	No refund by policy	—	—
Campaign send/DLR failure	Router campaign roll-up/refund workers	Campaign recipient / resend attempt	Router-defined campaign key
Kafka record routed to outbound DLT	Not implemented	—	—

The service registers and attempts a chunk refund after saving the dispatch result. It registers a recipient DLR refund in a separate transaction, and RefundRetryScheduler sends it later. By default, the scheduler checks PENDING records every 30 seconds. Every retry sends the same idempotency key to client-service. After ten failed HTTP calls, the record becomes FAILED and requires operator action.

The outbound internal call is:

POST http://client-service/internal/clients/{clientId}/balance/refund
X-Admin-API-Key: <shared key>

This is an API implemented by client-service, not by the aggregator.

Reporting correlation¶

Every recipient with a final status produces MESSAGE_FINALIZED. The aggregator does not send this event directly to Kafka. It adds the event to sms_router.outbox_event in the same transaction as the recipient update. The router's OutboxPublisher publishes it to sms.reporting, and client-service inserts or updates the reporting record by recipientId.

Important fields include the saved client and channel details, MSISDN, final status, carrier transaction IDs, CP and MNO response codes, cost, segments, and final timestamps. The copies of this event in the aggregator, router, and client service must keep the same fields.

This saves the final status and reporting event together. Reporting still depends on the router publisher working correctly.

Persistence and data ownership¶

Schema/table	Aggregator access	Purpose
`sms_router.sms_request_chunk`	Read/update	Dispatch claim, route used, carrier transaction, and chunk result.
`sms_router.sms_recipient`	Read/update	Send state, DLR schedule, final status, error, and cost for each recipient.
`sms_router.sender_config` / `sender_config_cli`	Read	Resolve active public/secondary route and CLI rule.
`sms_router.mno_user_config` / `mno_config`	Read	Credentials, billing identity, MNO type, TPS setting, and long-SMS capability.
`sms_router.outbox_event`	Insert	Final reporting event saved with the recipient state; the router publishes and manages it.
`sms_aggregator.aggregator_call_audit`	Insert	Infozillion send and DLR call history for investigation.
`sms_aggregator.balance_refund_log`	Read/write	Idempotent refund state and retry history.

This service's Flyway migrations manage only sms_aggregator. Router migrations must run first, and the database user needs explicit access to both schemas.

Audit, privacy, and observability¶

Send audit records store a masked recipient count and the CLI used. They do not store the message, credentials, API key, or full recipient list. They do store the response body, response time, response code, transaction ID, local exception class, and error message. DLR response bodies may contain MSISDNs in delivered, DND, or invalid lists. Access to this personal data must be restricted, and old data must be removed according to the retention policy.

Audit writing is best-effort. If JSON conversion or insertion fails, the service logs the error but continues dispatch. Reporting outbox writing is required: if it fails, the final recipient state is not committed without its reporting event.

Monitor Kafka lag, old DISPATCHING chunks, overdue SUBMITTED recipients, DLT volume, Infozillion response time and codes, 9099 causes, circuit state, pending or failed refunds, and router outbox backlog.

Scaling and failure isolation¶

A topic for each MNO user keeps one credential's backlog separate from others.
More consumer threads and service instances allow more sends at the same time, up to the number of partitions.
max.poll.records and max.poll.interval.ms must allow enough time for provider calls. Otherwise, Kafka may rebalance and deliver the event again.
Row locking prevents concurrent sends of the same chunk.
DLR workers share work through SKIP LOCKED.
Send and DLR use separate circuit breakers.
Refund retries depend on idempotency in client-service. The local query does not use SKIP LOCKED, so several service instances may try the same key at the same time.

Architecture decisions¶

Decision	Why it is used
Kafka-only business input	Provider response time does not block client request threads. The aggregator has no business API.
Per-MNO-user topics	Keeps each credential's backlog separate and routes work to the correct credential.
Exact outbound topic regex	Stops the main consumer from reading its own DLT and sending an SMS twice.
Save `DISPATCHING` before HTTP	Prevents a second send when the first result is unclear. A stuck claim must be checked manually.
Provider call outside the dispatch transaction	A slow send does not keep a database transaction open.
Select the final route during dispatch	Allows a configured primary or secondary route switch while saving both the selected and used route IDs.
Shared router execution tables	Lets API status and campaign roll-up observe the same recipient state.
Router outbox for final reports	Saves the final recipient state and reporting event in one transaction.
Choose refund owner by `campaignRef`	Stops both the aggregator and campaign process from refunding the same failure.
`NOT_AVAILABLE` terminal without refund	Avoids refunding a message that may have delivered without a final report.
Any `9099` billed and marked delivered	Applies the current Infozillion billing policy while keeping the cause in the audit.
Nine DLR checks with increasing delays	Limits provider load while checking status for about 23 hours.

Current limitations and operational risks¶

DltConsumer only reads and logs records. It does not create a refund or recovery task.
TpsLimiter exists and can be enabled as a bean, but MnpspDispatcher never calls it. tps_limit is not enforced. It is also JVM-local, not Redis/distributed.
No worker handles chunks left in DISPATCHING after a crash between the provider call and saving its result.
Actuator exposes *, and the service has no actuator security configuration. Restrict it by network policy or production override.
DLR polling keeps a database transaction open during provider calls. Large batches can use all database connections or hold locks for too long.
Audit response bodies have no service-level purge worker and may include MSISDNs.
Refund retries do not claim records before the HTTP call. Idempotency in client-service protects the balance, but several service instances may still do the same work.
Event classes are copied across services instead of being generated from one shared schema. Changes that do not match can break processing.

Aggregator Integration Service¶

System context¶

Responsibilities and exclusions¶

Runtime components¶

Inbound Kafka contract¶

Topic and consumer semantics¶

SmsChunkEvent¶

Effective sender routing¶

Infozillion integration contract¶

Send request mapping¶

DLR request and response¶

Dispatch consistency model¶

Submit outcome policy¶

The 9099 business decision¶

DLR state model¶

Refund ownership¶

Reporting correlation¶

Persistence and data ownership¶

Audit, privacy, and observability¶

Scaling and failure isolation¶

Architecture decisions¶

Current limitations and operational risks¶

`SmsChunkEvent`¶

The `9099` business decision¶