OperateTroubleshooting
Operate

Troubleshooting

Find and fix common problems with the install, sign-in, payments, sweeps, payouts and webhooks.

Markdown

Run each docker compose command as root in /opt/turmite.

Read the logs first

sudo docker compose ps
sudo docker compose logs --tail 200 api
sudo docker compose logs --since 1h worker | grep -i -E 'warn|error'
sudo docker compose logs caddy

Each log line is a JSON object. Errors name the network, the payment or the delivery that failed. See Monitoring and logs.

Install and Docker

Caddy cannot get a certificate

  • The DNS record must point to this server. Check with dig +short pay.example.com.
  • Ports 80 and 443 must be open to the internet. Check the provider firewall.
  • Behind Cloudflare, see Certificates behind Cloudflare.

Containers cannot resolve DNS

This happens with Tailscale or a local DNS stub (nameserver 100.100.100.100 or 127.0.0.53). Give Docker other resolvers:

echo '{ "dns": ["1.1.1.1", "9.9.9.9"] }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker

If daemon.json already exists, add the dns key to it. Do not overwrite the file.

docker compose cannot read .env

Only root can read /opt/turmite/.env. Run docker compose, backup.sh and restore.sh with sudo. Do not make .env readable for other users. It holds TURMITE_SECRET and the database password.

The install script stops

Message Fix
/opt/turmite/.env exists. Use --upgrade, or remove /opt/turmite to start again. Upgrade with --upgrade, or remove the directory
pass --site-url https://pay.example.com Give --site-url. Without a terminal, or with --yes, the script cannot ask
set TURMITE_HOT_PASSPHRASE, run in a terminal, or pass --yes Run the script in a terminal, or pass --yes for a random stored passphrase
this script supports Ubuntu and Debian; install Docker Engine and the Compose plugin yourself Install Docker, then run the script again
the API did not become healthy; see the logs above Read the log lines above the message. See The api exits at start

The api exits at start

Read the last lines of its log: sudo docker compose logs --tail 50 api. A start-up error is one line that starts with error:.

Message Fix
set TURMITE_SECRET (64 hex characters; ...) Set TURMITE_SECRET. turmite init writes one
TURMITE_SECRET must be hex or must be 64 hex characters (32 bytes) Use 64 hex characters, for example from openssl rand -hex 32
TURMITE_MODE must be mainnet or testnet Fix the value of TURMITE_MODE
this database was set up for testnet; TURMITE_MODE is mainnet Set TURMITE_MODE back to the mode of the database, or use a new database for the other mode
TURMITE_SITE_URL must be an http(s) URL or must not have a query or fragment Use a full http:// or https:// URL, without a query or a fragment
mainnet needs an https:// TURMITE_SITE_URL Use an https:// site URL on mainnet
TURMITE_TRUSTED_PROXIES: bad range Use CIDR ranges or IP addresses, separated by spaces
TURMITE_SMTP_URL: ... or TURMITE_SMTP_FROM: ... Fix the SMTP value. Percent-encode special characters in the URL
database error: ... Check TURMITE_DATABASE_URL and that postgres runs
internal error: migration failed: ... Restore the last dump before you try again, then report the error

The worker exits at start

Message Fix
set TURMITE_HOT_PASSPHRASE or TURMITE_HOT_PASSPHRASE_FILE to unlock the hot-wallet identity Give the passphrase. See Run without a stored passphrase
wrong hot-wallet passphrase Give the passphrase that you used with turmite init
read /secrets/hot-identity.age: ... The file is missing, or the worker user cannot read it. secrets/ must belong to 65532:65532
the hot-wallet identity cannot read the deposit seed (wrong identity file?) The identity in secrets/ does not belong to this database. Restore secrets/ from the same install as the database

To give secrets/ back to the worker user:

sudo chown -R 65532:65532 /opt/turmite/secrets

Sign-in

"Too many attempts. Wait a minute."

Sign-in allows 10 attempts per minute per IP address and per email address. Wait one minute.

If every user sees this message, Turmite sees one IP address for all visitors. Check TURMITE_TRUSTED_PROXIES and the proxy headers. See Client IP addresses.

Two-factor codes are refused

  1. Check the server clock: timedatectl. It must show System clock synchronized: yes.
  2. Turn on time sync if it is off: sudo timedatectl set-ntp true.
  3. Wait for the next code. Each code works one time.

After 5 wrong codes, Turmite refuses codes for that user for up to 15 minutes: "Too many wrong codes. Wait 15 minutes."

Lost authenticator device

The root user can click Reset 2FA for the user under Server → Users. Without the root account, run:

sudo docker compose exec api turmite user reset2fa [email protected]

The user signs in with the password and enrols again.

A form fails after sign-in

Turmite refuses a dashboard POST whose Origin (or Referer) header does not have the origin of the site URL. The answer is 403 Cross-site request refused.

  • Open the dashboard on the host of TURMITE_SITE_URL, or on a private entrance that rewrites Origin. See Private dashboard.
  • Check that your proxy does not remove or change Origin.
  • Check that TURMITE_SITE_URL has the correct scheme, host and path.

The session ends at once

On an https:// site URL, the cookie has the Secure flag. A browser does not send it over http://. Open the dashboard over https://.

Payments

A deposit does not show

  1. Open Server → Networks. The network must show Connected, and no No new block badge. If it does not, the nodes fail or rate limit you. Add a node.
  2. Check that the token is on for that network.
  3. Check that the customer used the correct network and the correct token contract.
  4. Search the worker log for the address or the transaction hash.

A payment stays in confirming

The deposit needs the confirmation count of the network. A Bitcoin block can take more than an hour. The checkout shows the count, for example 1 of 2. The defaults are in Networks and nodes.

The customer sent too little

The payment shows partially_paid. The checkout shows the missing amount. The customer can send the rest to the same address. Turmite watches the address until one day after the payment window closes. A payment that is still short then becomes expired. See Partial payments.

The customer paid after expiry

A deposit that arrives within one day after the payment window still counts. The payment continues to finished. The payment page in the dashboard shows the settlement Paid late. Check the price before you ship.

Turmite does not watch the address after that day. Check a later deposit on a block explorer and decide by hand: refund with a payout, or use Mark as paid.

The checkout status does not change

The status frame reloads every 4 seconds. Check the worker log for the payment. A browser extension that blocks frames can stop the reload. A page reload shows the current status.

Wallets

The first Bitcoin addresses do not match

Turmite derives native SegWit (BIP84) receive addresses from the account key. Use the account key of a BIP84 account: xpub or zpub on mainnet, tpub or vpub on Signet. A key of another account type gives other addresses.

Monero payments do not show

  • A restore height above the wallet's creation height hides older payments.
  • Check the private view key. It must belong to the primary address.
  • Check that the Monero node is synced.

An EVM factory deploy fails

The project's EVM hot wallet pays the gas for the deploy. Fund it on that network and try again.

Sweeps

Tron, Solana or TON funds stay at the deposit address

  1. Check the hot wallet balance on that network in Server → Hot wallets.
  2. Read the worker log for sweep errors.
  3. Check that the worker runs. Without the passphrase, the worker does not start, and nothing is swept.
Network Common cause
Tron Not enough TRX for energy and bandwidth on a USDT sweep
Solana Not enough SOL for fees, or for rent of a new token account
TON Not enough TON. The first sweep also deploys the deposit wallet

EVM funds stay in the deposit contract

The hot wallet needs gas for the flush. A token that is paused or that blocks the address can also fail. Funds in a deposit contract can only go to your cold wallet.

Payouts

A payout stays in pending

The worker signs payouts. Check that it runs and that it unlocked the hot identity:

sudo docker compose logs --since 1h worker | grep -i -E 'passphrase|identity|payout'

No approval email

  • Check TURMITE_SMTP_URL and TURMITE_SMTP_FROM.
  • Read the api log for SMTP errors.
  • Percent-encode special characters in the SMTP password.

A payout is in needs-review

Turmite cannot tell if the transaction landed. Check the hot wallet's history on a block explorer, then resolve the payout in the dashboard. Do not create a new payout before you know. See Payouts.

Webhooks

Every delivery fails

  • On mainnet, the URL must be public https. Turmite refuses private and reserved IP addresses.
  • Turmite does not follow redirects. Use the final URL.
  • Your endpoint must answer with a 2xx status.
  • Use Send test on the Webhooks page to try again.

The signature does not match

  • Use the correct secret. A project webhook endpoint signs with its own signing secret, which the Webhooks page shows one time, when you add the endpoint. The IPN of a payment (ipn_callback_url) signs with the IPN secret of the project.
  • For x-turmite-signature, compute the HMAC over the raw body bytes. Some frameworks parse the body first and change it.
  • For the HMAC-SHA512 signature, sort the keys of the JSON body before you sign. See Webhooks.
  • Read the name of the HMAC-SHA512 header in Settings → General. The default is x-turmite-sig.

The timestamp is outside your window

Your server clock is wrong. Turn on time sync: sudo timedatectl set-ntp true.

Pay with another coin

The checkout does not show "Pay with another coin"

  • The install runs on testnet. Trocador swaps mainnet coins only.
  • The exchange is off, or the project has no Trocador API key. See Pay with any coin.
  • The payment is not waiting or partially_paid, or its window closed.

Errors on the swap page

Message Cause What to do
The amount to send, ..., is less than the exchange minimum The amount is below the minimum for that coin The customer chooses another coin or pays directly
The amount to send, ..., is more than the exchange maximum The amount is above the maximum for that coin Same as above
The exchange has no quote for this coin No exchange with the KYC rating that you allow offers this pair now Allow a lower KYC rating, or the customer chooses another coin
The exchange cannot do this swap now Trocador refused the trade or did not answer Try again later. The customer can pay directly
The exchange returned no deposit address Trocador answered without an address Try again
The exchange cannot send this payment's coin Trocador does not list the payment's coin and network The customer pays directly. Or accept a coin that Trocador lists

Saving the API key fails

"The exchange did not accept the key" or "The exchange refused the API key" means that Trocador refused it. Check the key in your Trocador account. Turmite does not save a refused key.

The customer paid the exchange, but the payment is not paid

Turmite credits only what reaches the deposit address on chain. Open the payment in the dashboard and read its deposits. The customer's swap page shows the swap status and the swap ID. When the exchange sent less than the amount due, the payment is partially_paid. A refunded swap sends the coins to the refund address that the customer entered. Without a refund address, the customer contacts the exchange's support with the swap ID.

Database

Open a PostgreSQL shell:

sudo docker compose exec postgres psql -U turmite turmite

Do not change rows by hand unless a release note or a maintainer asks you to. Take a dump first.

Ask for help

Open an issue on GitHub. Add the Turmite version, the mode, the network and the log lines. Remove addresses and order IDs that you do not want to publish. Report security problems privately. See Reporting a vulnerability.