A2P SMS Message Identifiers: How to Normalize References Without Losing Traceability

The problem: one SMS can accumulate several references

A single logical send can generate several identifiers throughout its operational journey. The sending application may create its own reference; an HTTP API may return a resource ID; an SMSC or MC may return a message_id in submit_sm_resp; the provider may send a different reference in a DLR; and the callback receiver may assign its own event ID.

These references are not interchangeable. An identifier returned by a provider usually belongs to the scope of that platform, account, integration, and environment. It must not be treated as a global business primary key or assumed to be unique across providers, routes, accounts, or environments.

Risk arises when a system overwrites one ID with another, irreversibly transforms external values, or links events based only on a textual match. The result may be assigning an old DLR to a resend, mixing statuses from two providers, or losing the evidence needed to investigate an incident.

• The business event ID identifies the intent, such as an OTP request or an authorized transactional notification.
• The technical attempt ID identifies a specific sending execution to a particular account, provider, and route.
• The external ID identifies the resource or message within the system that issued it.
• The callback event ID identifies the received notification and not necessarily the SMS message it refers to.

Which identifiers can exist in HTTP, SMPP, and DLRs

The exact inventory depends on each integration contract, but it is useful to model stable categories. The goal is not to impose universal naming, but to record what each reference represents, who issued it, and in what context it can be used.

In SMPP, submit_sm_resp returns a message_id assigned by the MC or SMSC. The specification places it within the scope of the system that accepts the submit and allows it to be used in later operations, such as querying, replacing, or associating it with a receipt. It should therefore be retained as an external reference for the attempt, not as the global message identifier.

An SMPP receipt may arrive through deliver_sm or data_sm. When present, the receipted_message_id TLV carries the original message reference previously returned by the MC. Receipt data may also appear in other fields or formats defined by the integration. Store the PDU, or its raw representation, in addition to the normalized extraction.

In HTTP, a creation or acceptance response may return a message or resource identifier. Synchronous acceptance does not prove handset delivery. Later changes may arrive through a callback, a resource query, or a delivery report, each with its own event reference and timestamps.

• internal_event_id: immutable ID of the logical business event.
• send_attempt_id: immutable ID of each technical sending attempt.
• client_reference: optional reference provided by the client or originating system.
• external_message_id: ID returned by the provider, MC, SMSC, or API.
• dlr_reference: reference carried in the delivery report, such as receipted_message_id where applicable.
• callback_event_id: ID of the notification received by a webhook or event infrastructure.
• provider_account_scope: account, tenant, integration, environment, and provider that define the meaning of a reference.

Design principle: immutable internal IDs and versioned external references

The foundation of the design is simple: generate internal identifiers controlled by your organization and never reuse them. Then treat every external reference as evidence attributed to a source and a time of observation.

Separating the logical event from the technical attempt is essential. A business event can trigger an initial attempt, a controlled retry, or failover to another route. Each attempt must have its own send_attempt_id, even when all attempts belong to the same internal_event_id. This prevents a reissue from being interpreted as an update to the previous send.

External references must not be overwritten either. A single attempt may receive an acceptance reference, another reference in a DLR, and an additional reference in a later query. Record each as a separate row or event, with its type, original value, comparison value, and operational context.

Operational interpretation can evolve. For example, a derived status may move from pending to delivered or undelivered when new evidence arrives. However, the received event and the evidence that led to that interpretation must remain intact.

• Use UUIDs or another stable internal scheme for internal_event_id and send_attempt_id.
• Do not use a provider message_id as the primary key of the business domain.
• Maintain a one-to-many relationship between a technical attempt and external references.
• Maintain a one-to-many relationship between a technical attempt and status observations.
• Keep the integration version that processed each response or callback.
• Distinguish the observed status from the derived status used by your operations.

Practical normalization without destroying the received value

Normalization does not mean replacing the original value. The safe rule is to always store the exact representation received and, separately, create a comparison representation. This second representation is only for searches and documented linking rules.

Comparison may require examining encoding, length, whitespace, case sensitivity, prefixes, or truncation. Do not apply universal transformations: changing case may be harmless for one integration and destructive for another; trimming a string may create a collision; converting bytes to text without knowing the encoding may alter the identifier.

Every normalization must be reproducible. Store the rule name, its version, and the result. If the integration changes the format of a reference, you will be able to re-examine original values without losing evidence.

• external_value_raw: exact received value, preserved without transformation.
• external_value_compare: value derived for comparison under an explicit rule.
• normalization_rule_version: version of the applied rule.
• external_id_type: for example, submit_sm_resp_message_id, receipted_message_id, or http_message_id.
• observed_at: time when the system received or observed the value.
• source_payload_id: link to the stored payload, PDU, or raw event under controlled retention.
• Do not remove whitespace, zeros, prefixes, or non-alphanumeric characters without a provider-specific rule.

Minimum data model for investigation without overwriting evidence

A minimal relational model can support most investigations if it preserves the separation between intent, execution, references, and observations. It does not need to require all providers to return the same fields; it needs to explicitly record what was received and under what scope.

The business events table represents the authorized functional request. The attempts table represents each technical send. External references and status events relate to the attempt, not directly to the logical event, unless the provider contract makes that relationship demonstrable.

To minimize exposure, the destination must be treated as sensitive data. Record it in a consistent international form when needed for investigation and composite keys, with access controls, proportionate retention, and, where appropriate, tokenization or equivalent protection. It is not necessary to store full message content to resolve every incident; a secure hash of the payload or of a canonical representation can help distinguish attempts without unnecessarily increasing data exposure.

• business_event: internal_event_id, tenant_id, event type, idempotency_key, created_at.
• send_attempt: send_attempt_id, internal_event_id, provider_id, provider_account_id, route_id, environment, integration_version, submitted_at.
• external_reference: reference_id, send_attempt_id, external_id_type, raw_value, compare_value, normalization_rule_version, observed_at.
• status_observation: observation_id, send_attempt_id, callback_event_id, raw_status, normalized_status, provider_timestamp, received_at, payload_reference.
• investigation_context: protected or tokenized destination, secure payload hash, sending source, and required audit data.

When the provider reuses, transforms, or does not return a correlatable reference

Not every provider preserves a client-supplied reference, returns a stable ID, or includes the same ID in DLRs. The model must accommodate this limitation without inventing a relationship that cannot be proven.

If a provider reuses identifiers, the reference can only be unique within a composite key. At a minimum, include tenant, provider, provider account, environment, reference type, and an observation time window. Add route and integration when they can change the operational meaning of the value.

If the provider transforms the identifier, record both values and the known transformation rule. If no contractual or technically verifiable rule exists, do not create an automatic association based on partial similarity. Mark the case as ambiguous and send it for reconciliation or investigation.

When no correlatable reference exists, traceability can continue as far as the technical attempt and acceptance evidence, but the link to a specific DLR will remain uncertain. That limit must be visible in the dashboard and operational procedures.

• Never deduplicate globally using an isolated external_message_id.
• Do not use prefix, suffix, or truncation matches as proof of identity.
• Require a composite key with operational scope for every lookup rule.
• Classify links as confirmed, probable, or non-correlatable; reserve irreversible automations for confirmed links.
• Document which references each provider returns and which may appear in its DLRs.

submit_sm_resp, DLRs, and asynchronous statuses: what can be linked

In a typical SMPP flow, the ESME sends submit_sm and receives submit_sm_resp. The response message_id identifies the message in the MC or SMSC that responded. If a receipt has been requested through registered_delivery and the system emits a DLR, the receipt may indicate that it is an MC Delivery Receipt through esm_class and carry the received message identifier in the receipted_message_id TLV.

This relationship enables strong correlation when receipted_message_id matches the previously recorded message_id within the same provider, account, environment, and integration. Even so, retain the full DLR: its status, timestamps, and available fields are part of the evidence and may be needed if duplicates or out-of-order events occur.

In HTTP, the identifier returned when a resource is created may be used to query its status later or link callbacks, depending on the provider contract. An HTTP creation or acceptance status indicates that the platform processed or queued the request according to its semantics; it does not independently prove handset delivery.

Callbacks may arrive late, duplicated, or out of order. Do not automatically discard an observation simply because it is old relative to its receipt time. Compare the provider timestamp, receipt timestamp, and known sequence; then apply auditable closure and reconciliation rules.

• Persist submit_sm, submit_sm_resp, and DLRs as separate stages.
• Request DLRs through registered_delivery when the SMPP contract and use case require them.
• Do not treat a DLR as proof of human reading or overall route quality.
• Do not assume that a terminal status prevents later contradictory or duplicate evidence from arriving.
• Maintain a documented policy for deciding which derived status is shown, without deleting previous statuses.

Route changes, resends, and duplicates: model by context

Failover, a retry, or a reissue may relate to the same business event, but they are not necessarily the same technical message. The practical rule is to create a new send_attempt_id for every send to a specific combination of tenant, provider, account, route, environment, and integration.

Do not consolidate DLRs from different routes as updates to the same attempt. A status from a previous route must not be assigned to a new route merely because the destination, content, or an external reference appears similar. The correct relationship is maintained through internal_event_id, while each provider's evidence remains associated with its own attempt.

Idempotency must be applied before sending. For HTTP requests, POST is not idempotent by definition; retrying after an uncertain response can duplicate a send if there is no application idempotency key or reliable confirmation that the earlier operation was not applied. Do not rely on an external ID that may not yet have been returned.

A deliberate resend must also be visible as such. Record the reason: acceptance timeout, technical failure, failover policy, manual decision, or another authorized reason. This makes it possible to distinguish accidental duplication from a controlled second execution.

• Recommended context key: tenant_id, provider_id, provider_account_id, route_id, environment, external_id_type, and external_value_compare.
• Add time windows only as an additional constraint, not as the sole proof of identity.
• Use an idempotency_key per business event or intent before invoking the provider.
• Record retry_sequence, failover_reason, and the relationship between the source attempt and successor attempt.
• Avoid sending unnecessary sensitive content to logs, search tools, or URLs.