A stub in place of a real SMS code checks the form, not the whole scenario: it does not catch delivery delay, an unexpected code format from a specific service, or a break at the confirmation-email step. A run with a genuine code is the only way to confirm the registration chain is alive end to end, from the first number request to logging into the account.
Why a real-code scenario matters
A mock code always arrives instantly and in the expected format — reality is different. A carrier can delay delivery by 20–40 seconds, a code can arrive as two SMS instead of one, a confirmation email can sit in a moderation queue on the mail service's side. None of that is reproduced by a stub, and these are exactly the failures that tend to break a production registration flow no one has touched in months.
The second reason is the integration as a whole. A real-code test checks the entire chain: requesting a number, delivering the code, extracting it from the text, entering it in the form, finalizing the activation. A break at any link is a bug worth catching in CI, not in production.
How the end-to-end run is structured
The scenario rests on three steps. First, requesting a number or a mailbox through the API before starting registration on the target platform. Second, waiting for the code: it does not arrive instantly, so the pipeline polls the status at a fixed interval rather than relying on a single blind timeout. Third, finalization: the code is entered into the form, registration completes, and the number or address is marked used so a repeat request is not sent for a code that will never come.
The wait timeout needs margin over normal delivery: if a code typically arrives in 10–15 seconds, a 60–90 second timeout is reasonable, not 15. Retrying the request makes sense once, after the first timeout expires — a second attempt catches an occasional carrier delay, while a third usually signals a real break in the chain rather than delivery jitter.
Why this stays a separate, rare run rather than part of every commit
A real-code test costs money on every run — the number or address is billed regardless of outcome — and depends on an external service whose delay is outside your control. Running it on every commit means paying for every push and getting random red builds caused not by the code but by an external delivery delay.
The working pattern is to keep fast stub-based tests in the regular per-commit pipeline and move the real-code scenario into a separate, rare run: before a release, once a day on a schedule, or manually before merging to the main branch. That catches real regressions in the delivery chain without inflating the cost of the ordinary build.
Isolating test data from production
Numbers and mailboxes for automated tests should come from a separate pool, not the same balance used for production tasks: mixing them makes cost tracking harder and risks a test run accidentally grabbing a number reserved for a live process. Accounts created by a test are worth tagging with a distinct prefix or mail domain and cleaning up on a schedule — otherwise test records pile up in the same tables as real users and confuse analytics.
Accounting for run costs
Every real-code run is a charge for the number or mailbox plus pipeline execution time. Calculate the cost of one full scenario and multiply by run frequency: once a day is one order of budget, on every commit in an active branch is several times higher without a proportional gain in value. Keep real-code runs frequent enough to catch regressions before release, yet rare enough that CI spend does not outrun the cost of the service itself.
Frequently Asked Questions
Can stubs be dropped entirely in favor of real codes?
There is no point: a stub is fast, free, and good for testing parsing logic and the form on every commit. A real code is needed for a separate end-to-end scenario that catches integration and delivery problems, not for testing application code on every edit.
What should happen if the code does not arrive within the timeout?
One retry of the request is a reasonable first step — it absorbs an occasional carrier delay. If the code still does not arrive after the retry, the test should fail explicitly and loudly rather than hang until the pipeline's overall timeout — that saves triage time and points clearly at a delivery-chain problem rather than jitter.
How often should a real-code run happen?
A practical cadence is once a day on a schedule plus a mandatory run before a release to the main branch. That rhythm catches regressions within a reasonable window and keeps the spend on numbers and addresses predictable, independent of commit volume during active development.
How the number request and code-status polling work is covered in the API documentation. Get a number for the first test run in the OTP section — the polling principle is the same one used for rotating IPs via the API. Track spend on runs in the transaction history.