Resources · automationreliability
Your automations are failing right now and nobody knows
Some of your automations are probably failing right now, and you have not noticed. That is normal, because automations almost never crash. They drift.
The zap keeps reporting "success" while a change upstream quietly turned its output wrong, and that can run for weeks before anyone catches it. This post covers why silent failure happens, the three shapes it takes, and a 15-minute check you can run this month to find it.
Why don't broken automations tell you they broke?#
Because most of them never actually break. They keep running on inputs that changed underneath them.
A form adds one new field. An export renames a column. An API lowers a rate limit, or a login flow adds a step. None of that throws an error. The automation still fires, still finishes, still writes a green "success" in the log. It is just doing the wrong thing now, and the log has no way to know that.
What actually costs you is the weeks between the day it went wrong and the day a human finally looked. Every one of those days shipped bad data to a client, a report, or a decision, at full speed, with a success stamp on it.
What are the three ways an automation fails?#
There are three, and most monitoring only catches the first one.
| Failure type | What it looks like | How you usually find out |
|---|---|---|
| Dead | It stopped running entirely | Eventually. A person notices a thing that never arrived |
| Drifting | It still runs and still reports success, but an input changed and the output is now wrong | Weeks later, from a mistake downstream |
| Confidently wrong | An AI step fills a gap with a clean, plausible, wrong answer | When a client catches it before you do |
Dead is the easy one. It is loud in its own way, because something obvious goes missing. Drifting is the expensive one, and it is exactly why silence is not the same as success. The success log is measuring "did it run," not "was it right."
The third shape is newer. A plain Zapier zap fails quietly when its inputs change. An AI step fails a different way: it invents a clean, wrong answer instead of stopping. Both need watching, for different reasons and in different ways, and neither one warns you.
How do I check if my automations are failing right now?#
Run this once a month. It takes about fifteen minutes and a VA can do all of it without touching any settings.
| Check | How to run it | What means trouble |
|---|---|---|
| Volume, month over month | Pull the last 30 days of runs, compare the count to the month before | A drop of around 40% that nobody planned - something upstream changed |
| Output, end to end | Open three finished runs and follow each one to its real result | The run says success but the output is stale, blank, or wrong |
| Alerts reach a human | Trigger one test failure and see where the notice lands | The alert goes to a channel or inbox nobody reads |
| Last human look | List every automation and the date a person last checked its output | Anything past 60 days means nobody has confirmed the output is still right |
The volume check is the one that finds drift the fastest. A quiet automation and a broken one produce the same number: fewer outputs than last month. You do not need to know why yet. You just need the number to jump out at you.
How do we build a watcher into every system?#
We assume every automation we ship will drift one day, so we build a second thing whose only job is to notice.
Every automation reports what it did to one Slack channel: how many items it processed, and a one-line summary of the result. That is the diary. Then a second, much smaller check reads those diary entries on a schedule and looks for two things only: did any automation go quiet, and did any automation's volume roughly halve. When it spots either, it flags a human. A person reads the flag and decides if it is real.
The watcher is deliberately dumber than the automation it watches. It only knows what "normal volume" looked like last week and whether this week is far off that. Smart watchers fail the same silent way the automations do, so we keep this layer simple enough that it almost cannot. This is the same detection idea behind automation whose real job is finding problems, pointed back at your own systems instead of your accounts.
The client keeps all of it in plain sight: the diary channel, the list of what each automation is supposed to do, and the rule for what counts as "too quiet." When you go shopping for an automation partner, ask them what happens when a build breaks at 2am and who finds out first. A shop that has run automations in production will already have a watcher. A shop that has not will tell you their builds "just work," which is the answer that costs you the silent weeks.