How I Built a Zero-Collision, Cryptographically Obfuscated URL Shortener in Go
When you think of building a URL shortener, the standard textbook approach is simple: generate a random string, check if it exists in the database, insert it if it doesn’t, and retry if there’s a collision.
But in a production environment, this approach falls apart at scale. Check-then-insert loops lead to expensive database round-trips, random generation leads to predictable hash collisions as your data grows, and vulnerable sequential database IDs expose your service to URL enumeration attacks.
To solve this, I designed and built a production-grade URL shortener in Go that guarantees zero database collisions by design, obfuscates sequence keys using cryptography, and implements advanced Site Reliability Engineering (SRE) patterns to shield the databases under heavy load.
In this article, I’ll walk you through the system design, the mathematics behind collision-free IDs, and the SRE strategies I used to make this system resilient.
The System Architecture
Unlike standard services, this URL shortener separates the write path and read path entirely, ensuring each concern is optimized for throughput.
Most URL shorteners struggle with collisions. To guarantee zero collisions without any lookup-and-retry loops, I implemented a bijective cryptography pipeline:
Step A: PostgreSQL Sequences
When a URL is shortened, PostgreSQL allocates a unique, auto-incrementing 64-bit integer (BIGINT). Because PostgreSQL handles sequence allocation atomically, we are guaranteed to start with a perfectly unique number.
Step B: The 4-Round Feistel Cipher
Exposing sequential IDs directly (e.g., example.com/1, example.com/2) is a massive security risk. Malicious actors can easily scrape your entire database.
To solve this, the sequential ID is run through a symmetric 4-Round Feistel Cipher using a private 32-bit seed. A Feistel cipher is a block cipher structure. Because it is mathematically bijective (a perfect one-to-one mapping), it guarantees that every input number maps to a completely distinct obfuscated number.
- Result: No two sequential IDs will ever produce the same obfuscated output, and yet it is impossible to guess the next code in sequence.
Step C: Base62 Web-Safe Encoding
Finally, the scrambled 64-bit ID is translated into a Base62 string using the character set [a-zA-Z0-9]. This results in highly compact short codes of up to 11 characters.
2. High-Performance Caching & Rate Limiting
To handle viral redirects with sub-millisecond latencies, I integrated Redis into the hot paths using the Repository Decorator Pattern:
Cache Pre-warming: When a URL is successfully shortened, it is immediately written to Redis. It is “pre-warmed” before any user even tries to access it.
Bijective Expire Synchronization: Cache lifetimes (TTLs) are synchronised dynamically to match the precise database
ExpiresAtvalue.Distributed Rate Limiting: To prevent API abuse, a custom middleware tracks IP addresses. It runs atomic Redis transactions (
TxPipeline) to evaluate fixed-window rates in a single round-trip, keeping Go instances fully stateless and scalable.
3. Advanced SRE Production Shielding
As I prepared the codebase for real-world scaling, I integrated several core Site Reliability Engineering (SRE) patterns:
Eliminating Cache Stampedes (Thundering Herd)
Under peak concurrent loads, if a highly popular cached URL expires or is dropped, thousands of concurrent Go routines will experience a cache miss simultaneously. If left unhandled, they will all storm the PostgreSQL database, saturating connection pools and spiking CPU.
I protected the database using Go’s Singleflight pattern (golang.org/x/sync/singleflight):
dbVal, err, _ := r.sfGroup.Do(code, func() (interface{}, error) {
return r.postgresRepo.GetByShortCode(ctx, code)
})
If a cache miss occurs, the single-flight block ensures that only one active query goes to PostgreSQL. All other waiting concurrent requests block and share the returned in-flight value, saving database resources.
Composite Health Checks (GET /health)
A health check shouldn’t just check if the Go server is running. The health check queries both PostgreSQL and Redis concurrently. If either backend fails, it returns a detailed component breakdown alongside a 503 Service Unavailable status code, prompting load balancers to instantly isolate the node before requests are lost.
Docker Container Resource Tuning
To prevent memory leaks or an unconstrained Redis container from crashing the host machine, I added dynamic memory caps (REDIS_MAX_MEMORY=256mb) and configured an allkeys-lru eviction policy. If Redis reaches its memory ceiling, it automatically evicts the least recently used keys, keeping the container completely stable.
Tech Stack
Language: Go 1.22+ (Native, fast, and light memory footprint)
Routing: Go Chi v5
Relational DB: PostgreSQL 17
In-Memory Store: Redis 7.4
What’s Next for Me?
Building this project taught me how valuable robust system design is. But software development doesn’t stop at writing the code.
I’ve decided to hit the books on DevOps next! As I continue to learn and grow, my goals for improving this project are:
Containerization: Mastering Docker and complex multi-container networking.
Dynamic Deployments: Automating local setups using tuned Docker Compose environments.
CI/CD Pipelines: Setting up GitHub Actions to build, lint, test, and package Go images automatically.
Infrastructure as Code (IaC): Provisioning cloud SQL databases, Redis cache instances, and serverless compute nodes securely using Terraform.
Monitoring & Logging: Hooking up Prometheus, Grafana, or an ELK stack to ingest structured JSON logs and trace system behaviour
Check Out the Code
The complete repository is open-source, featuring recursive environment loading, migration scripts, and fully mocked unit/integration test suites.
