Systems engineering, under live load
Fix a real system while it's on fire.
Crucible drops you into a running distributed system that is misbehaving under real traffic. Read the metrics, change the code and config, ship the fix, and let a hidden judge decide whether it held.
- 4-phase
- implement → dev → test → judge
- real
- containers, not mocks
- hidden
- judge replays your fix
- p99 latency
- 1.8 s
- Error rate
- 4.2%
- Cache hit
- 0%
How it works
One loop, against a live system
Implement
Read the symptom, inspect the architecture and logs, then edit the service code and configuration in the browser.
Dev
Bring your environment live. Your build runs as real containers against a real database, not a mock.
Test
Drive load through the system and watch latency, errors and throughput respond to your change.
Judge
Submit. A hidden judge replays traffic and scores whether the fault is actually fixed and the system stays healthy.
Example scenario
The Cache Layer
Product pages are slow and occasionally erroring under load. Every read hits Postgres directly — there is no cache in the path. Find the bottleneck and fix it.
- caching
- http
- postgres
- latency
What you will do
- Find why product reads saturate Postgres under load
- Put a cache in the read path without serving stale data
- Hold latency and error targets while traffic keeps flowing
Ready to fix something that is actually broken?
No setup, no local cluster. Pick a scenario and your environment comes up in the browser.